-- Logs begin at Thu 2022-06-23 08:26:16 UTC, end at Thu 2022-06-23 09:44:20 UTC. -- Jun 23 09:02:28.423809 dv-cinder-847274-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 23 09:02:28.431168 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 23 09:02:28.433104 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 23 09:02:28 2022] *** Jun 23 09:02:28.433104 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 23 09:02:28.433104 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 23 09:02:28.433104 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: nodename: dv-cinder-847274-1-iscsi Jun 23 09:02:28.433104 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: machine: x86_64 Jun 23 09:02:28.433104 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: clock source: unix Jun 23 09:02:28.433104 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: pcre jit disabled Jun 23 09:02:28.433104 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: detected number of CPU cores: 4 Jun 23 09:02:28.433632 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: current working directory: / Jun 23 09:02:28.433632 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: detected binary path: /usr/bin/uwsgi-core Jun 23 09:02:28.433632 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: your processes number limit is 47680 Jun 23 09:02:28.433632 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: your memory page size is 4096 bytes Jun 23 09:02:28.433632 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: detected max file descriptor number: 2048 Jun 23 09:02:28.433632 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: lock engine: pthread robust mutexes Jun 23 09:02:28.433632 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: thunder lock: enabled Jun 23 09:02:28.433632 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 23 09:02:28.433632 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 23 09:02:28.479933 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: Python main interpreter initialized at 0x5583a1ffe9c0 Jun 23 09:02:28.479933 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: python threads support enabled Jun 23 09:02:28.479933 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: your server socket listen backlog is limited to 100 connections Jun 23 09:02:28.479933 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: your mercy for graceful operations on workers is 90 seconds Jun 23 09:02:28.480311 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: mapped 403077 bytes (393 KB) for 2 cores Jun 23 09:02:28.480391 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: *** Operational MODE: preforking *** Jun 23 09:02:28.480660 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: *** uWSGI is running in multiple interpreter mode *** Jun 23 09:02:28.480660 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: spawned uWSGI master process (pid: 80644) Jun 23 09:02:28.480864 dv-cinder-847274-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 23 09:02:28.481673 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: spawned uWSGI worker 1 (pid: 80654, cores: 1) Jun 23 09:02:28.482340 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: spawned uWSGI worker 2 (pid: 80655, cores: 1) Jun 23 09:02:28.482502 dv-cinder-847274-1-iscsi devstack@keystone.service[80644]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 23 09:02:33.980141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 09:02:33.980141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: from cryptography.utils import int_from_bytes Jun 23 09:02:33.980955 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 09:02:33.980955 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: from cryptography.utils import int_from_bytes Jun 23 09:02:33.981374 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 09:02:33.981374 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: from cryptography.utils import int_from_bytes Jun 23 09:02:33.981489 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 09:02:33.981489 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: from cryptography.utils import int_from_bytes Jun 23 09:02:34.030176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 09:02:34.030375 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 09:02:34.030538 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] command line args: [] {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 09:02:34.030814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 09:02:34.031020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ================================================================================ {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 09:02:34.031280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] admin_token = **** {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.031531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] config_dir = [] {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.031721 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.031807 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 09:02:34.031891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] config_source = [] {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.031965 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 09:02:34.032072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] debug = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.032165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] command line args: [] {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 09:02:34.032270 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.032540 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 09:02:34.032540 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ================================================================================ {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 09:02:34.032666 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.032810 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] admin_token = **** {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.032890 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.033037 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] config_dir = [] {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.033124 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.033241 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.033325 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.033400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] config_source = [] {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.033511 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] list_limit = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.033592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] debug = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.033701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] log_config_append = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.033804 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.033939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.034045 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.034156 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] log_dir = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.034261 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.034365 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] log_file = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.034469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.034593 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.034681 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.034761 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.034886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] list_limit = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.035008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.035097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] log_config_append = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.035178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.035254 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.035380 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.035465 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] log_dir = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.035544 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.035673 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] log_file = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.035751 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.035873 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.035975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.036062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.036211 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.036298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.036447 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.036527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.036623 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.036708 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.036793 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.036903 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.037033 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.037150 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.037283 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.037398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.037527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.037679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.037813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.037989 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.038176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.038305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.038387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] publish_errors = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.038466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.038585 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.038701 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.038793 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.038993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.039087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.039244 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.039480 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.039565 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.039672 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] publish_errors = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.039751 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] standard_threads = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.039909 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.040009 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.040146 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.040234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.040311 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.040389 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.040493 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] use_journal = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.040572 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.040668 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] use_json = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.040785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.040871 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] use_stderr = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.041003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] use_syslog = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.041158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] standard_threads = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.041235 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.041380 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.041481 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.041639 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.041725 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.041807 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] use_journal = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.041882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.041985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] use_json = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.042137 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.042223 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] use_stderr = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.042322 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] use_syslog = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.042413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.042557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 09:02:34.042643 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.042816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.042895 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.043081 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.043173 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.043345 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.043432 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.043588 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.043669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.043835 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.043923 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.044073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.044646 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.044830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.044915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.045072 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.045160 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.045309 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.045388 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.045559 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.045639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.045808 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.045903 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.046046 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.046126 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.046290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.046369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.046529 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.046620 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.046779 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.046871 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.047045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.047126 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.047284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.047447 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.047530 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.047610 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.047784 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.047870 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.048034 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.048131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.048421 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.048515 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.048591 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.048660 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.048774 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.048856 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.049013 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.049100 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.049265 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.049351 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.049515 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.049595 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.049748 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.049828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.049980 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.050060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.050218 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.050304 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.050467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.050545 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.050707 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.050818 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.050947 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.051063 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.051186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.051301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.051442 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.051550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.051659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.051766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.051909 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.052051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.052183 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.052308 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.052476 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.052602 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.052730 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.052853 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.052986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.053110 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.053247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.053379 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.053513 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.053638 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.053763 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.053888 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.054019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.054156 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.054293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.054370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.054514 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.054644 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.054775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.054943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.055082 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.055208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.055359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.055491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.055575 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.055748 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.055748 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.055998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.056122 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.056249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.056386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.056518 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.056646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.056769 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.056903 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.057029 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.057158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.057293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.057420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.057546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.057686 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.057822 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.057986 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.058120 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.058251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.058378 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.058505 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.058633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.058783 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.058951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.068043 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.068221 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.068364 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.068497 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.068639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.068845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.068976 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.069104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.069240 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.069369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.069502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.069639 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.069767 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.069896 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.070033 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.070167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.070304 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.070449 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.070584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.070722 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.070851 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.071035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.071174 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.071310 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.071442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.071579 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.071710 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.071837 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.071970 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.072099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.072225 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.072369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.072499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.072626 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.072759 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.072887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.073013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.073148 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.073276 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.073403 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.073530 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.073657 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.073784 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.073918 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.074057 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.074185 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.074311 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.074454 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.074587 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.074726 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.074858 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.075008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.075134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.075259 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.075383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.075518 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.075648 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.075782 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.075918 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.076048 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.076177 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.076311 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.076441 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.076566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.076690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.076815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.076948 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.077073 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.077198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.077322 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.077451 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.077576 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.077717 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.077843 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.077969 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.078098 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] auth.external = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.078223 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.078347 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.078482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.078608 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.078734 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.078866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.079080 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.079209 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] auth.password = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.079376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] auth.external = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.079523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] auth.token = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.079662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.079791 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.079918 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.080045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.080173 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.080306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.080441 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] auth.password = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.080575 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.080708 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] auth.token = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.080837 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.080967 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.081069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.081249 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.081639 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.081773 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.081903 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] credential.caching = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.082031 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.082160 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.082294 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.082428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.082559 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.082686 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.082813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.082952 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.083088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] credential.caching = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.083214 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.083349 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.083486 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.083618 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.083743 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.083877 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.084004 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.084129 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.084272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.084397 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.084528 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.084736 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.084864 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.084991 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.085117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.085253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.085253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.085497 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.085624 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.085751 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.085751 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.085987 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.086112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] federation.caching = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.086249 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.086375 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.086500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.086625 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.086751 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.086885 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.087079 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.087079 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] federation.caching = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.087289 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.087451 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.087577 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.087707 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.087844 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.087971 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.088107 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.088241 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.088374 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.088508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.088644 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.088786 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.088915 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.089043 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.089170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.089305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.089434 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] identity.caching = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.089563 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.089696 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.089823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.089951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.090086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.090214 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.090341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] identity.caching = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.090467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.090604 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.090731 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.090866 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.091009 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.091140 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.091265 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.091395 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.091525 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.091665 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.091797 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.091926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.092056 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.092193 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.092341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.092341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.092595 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.092725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.092855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.092980 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.093110 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.093110 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.093367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.093496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.093625 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.093754 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.093955 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.094096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.094233 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.094366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.094491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.094616 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.094747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.094885 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.095061 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.095061 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.095314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.095444 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.095571 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.095705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.095834 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.095971 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.096099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.096184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.096393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.096393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.096658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.096775 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.096902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.097032 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.097159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.097296 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.097427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.097558 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.097688 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.097827 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.097955 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.098094 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.098223 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.098350 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.098479 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.098606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.098734 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.098734 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.099002 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.099128 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.099253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.099253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.099499 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.099640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.099766 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.099910 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.100036 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.100162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.100288 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.100421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.100548 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.100674 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.100800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.100925 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.101056 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.101188 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.101325 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.101451 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.101577 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.101703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.101837 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.101964 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.102089 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.102214 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.102339 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.102464 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.102598 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.102725 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.102850 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.user = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.102993 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.103134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.103264 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.103401 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.103531 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.103660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.103788 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.user = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.103922 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.104052 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.104194 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.104324 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.104451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.104584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.104714 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.104854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.104991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.105119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.105246 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.105373 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.105500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.105629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.105764 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.105893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.106020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.106148 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.106280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.106407 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.106542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.107228 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.107355 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.107480 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.107605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.107738 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.107872 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.107999 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.108124 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.108249 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.108249 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.108484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.108617 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.108748 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.108875 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.109010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.109137 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.109262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.109395 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.109522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.109647 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.109772 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.109897 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.110023 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.110225 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.110358 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.110491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.110618 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.110754 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.110880 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.111040 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.111165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.111293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.111420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.111547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.111673 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.111809 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.111936 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.112063 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.112192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.112319 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.112447 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.112595 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.112724 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.112856 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.112984 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.113111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.113238 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] resource.caching = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.113373 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.113501 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.113629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.113755 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.113882 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] resource.caching = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.114010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.114146 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.114276 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.114411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.114539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.114673 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.114800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.114953 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.115080 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.115206 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.115332 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.115458 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.115583 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.115720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.115846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] role.caching = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.115971 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.116097 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] role.driver = sql {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.116236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.116362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.116496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] role.caching = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.116622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.116749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] role.driver = sql {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.116874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.117000 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.117133 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.117259 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.117386 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.117511 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.117637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.117762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.117905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.118032 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.118157 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.118282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.118408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.118534 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.118667 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.118794 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.118971 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.119098 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.119228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.119357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.119497 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.119635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.119765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.119899 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.119899 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.120142 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.120278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.120407 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.120407 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.120651 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.120779 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.120908 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.121047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.121176 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.121315 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.121444 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.121572 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.121700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.121836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.121967 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.122096 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.122224 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.122383 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.123027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.123156 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.123283 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.123408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.123544 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.123671 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.123806 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.123933 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.124059 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.124059 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.124295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.124421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.124556 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.124556 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.124796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.124921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.125047 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.125172 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.125315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.125442 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] token.caching = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.125569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.125695 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.125822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.125947 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.126081 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] token.caching = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.126207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.126333 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.126459 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.126586 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.126586 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.126831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.126831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.127096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.127227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.127362 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.127490 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.127631 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.127631 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.127876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.128005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.128134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.128134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.128384 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.128512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.128640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.128640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.128890 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.129020 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.129159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.129287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.129415 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.129542 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.129670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.129797 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.129935 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.130064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.130190 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] database.connection = **** {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.130329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.130465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.130603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] database.connection = **** {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.130741 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.130879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.131026 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.131026 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.131349 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.131479 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.131609 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.131739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.131875 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.132014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.132146 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.132274 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.132404 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.132541 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.132670 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.132813 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.132944 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.133080 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.133211 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.133341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.133468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.133606 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.133743 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.133872 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.134001 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.134129 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.134257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.134402 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.134532 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.134659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.134786 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.134974 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.135103 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.135245 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80655) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 09:02:34.135375 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.135375 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 09:02:34.135375 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 09:02:34.148496 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.148606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.148732 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.148808 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.148955 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.149033 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.158030 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.158139 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.186303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.186448 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.186534 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.186636 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.328383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.329685 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.376902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.377133 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.379259 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.379259 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.455605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.455833 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.455926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.456058 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.456148 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.456276 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.456615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.456755 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.456840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.456970 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.457056 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.457201 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.482854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.483051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.483164 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.483285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.483372 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.483507 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.483592 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.483716 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.483802 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.483929 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.484014 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.484141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.484224 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.484353 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.495221 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.495358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.507902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.508138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.508242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.508364 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.508449 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.508583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.508668 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.508796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.508883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.509007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.509093 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.509216 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.509307 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.509428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80655) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 09:02:34.526841 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.527078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 09:02:34.527412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.527496 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.527612 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 09:02:34.527698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 09:02:34.527874 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.527991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.528130 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.528224 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 09:02:34.528387 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.528477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.529224 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.529320 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.529503 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.529757 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.539997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.540276 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.540515 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.540601 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.540788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.540874 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.541053 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.541328 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.549093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.549631 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80655) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.552401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.552684 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.552922 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80655) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.553007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.553203 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.553300 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.553479 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.553563 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.553732 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.553816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.553996 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.554082 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.554252 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.554335 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.554514 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.554605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.554776 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.554860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.555066 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.555150 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.555334 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.555596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.582282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.582955 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.583876 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.584154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.584559 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80655) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.584833 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.589409 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.589797 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.590074 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.590261 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80655) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.590653 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.590948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.597417 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.597698 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.598261 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80655) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.598640 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.602175 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.602458 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.602734 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.603035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.603234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80655) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.603320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.603517 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.603602 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.603795 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.603879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.604060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.604144 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.604332 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.604599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.604859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.605120 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.623319 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.623589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.624362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.624634 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.628242 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.629347 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80655) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.631871 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.632262 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.632995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.633385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.637287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.638937 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80655) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.641432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.642532 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80655) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.645099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.645402 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.645685 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.645967 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.646196 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.646283 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.646491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.646577 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.646772 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.647078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.647358 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.647632 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.665570 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.665853 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.666115 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.666386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.666770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80655) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.667075 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.667347 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.667604 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.674806 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.676227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.676528 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.676813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.677929 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.678199 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.680012 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.680291 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.680565 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.680839 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.681125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.681541 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80655) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.681836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.682134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.682437 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.682746 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.694462 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.694738 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.695056 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.695337 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.696203 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80655) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.696483 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.696766 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.696858 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.697045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.697130 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.697398 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.698550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.698837 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.699153 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.716071 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.718776 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80655) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.719912 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.722080 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80655) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.723272 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.725363 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.727476 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.728346 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.728820 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.729286 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.730414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.733211 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80655) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.733502 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.733780 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.739479 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.741102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.741351 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.742969 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80655) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.744444 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.744732 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.745123 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.745399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.746342 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.746643 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.747060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.747372 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.754685 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.754999 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.755276 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.756421 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80655) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.756706 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.756987 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.763675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.763954 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.764231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.764501 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.764765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.765129 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.765406 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.765538 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80655) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 09:02:34.765669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.765799 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.765940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.766077 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.766214 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.766355 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.766483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.766612 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.766740 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.766877 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.767035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.767162 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.767295 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.767435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.767682 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.767952 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.768220 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.768855 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.769144 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.769411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80655) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 09:02:34.805795 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5583a1ffe9c0 pid: 80654 (default app) Jun 23 09:02:34.807859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5583a1ffe9c0 pid: 80655 (default app) Jun 23 09:02:34.809280 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d770a3a-1845-4215-812a-348daa887d2b None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:34.809574 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d770a3a-1845-4215-812a-348daa887d2b None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:34.810395 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d770a3a-1845-4215-812a-348daa887d2b None None] PATH_INFO: `/v3` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:34.814615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Thu Jun 23 09:02:34 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:02:43.578974 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3b756-c48b-4084-9738-d59d3d3cf166 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:43.579495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3b756-c48b-4084-9738-d59d3d3cf166 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:43.579603 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3b756-c48b-4084-9738-d59d3d3cf166 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:43.584100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:02:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:02:43.589404 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c50db9-2b8c-458b-a8a0-0334ba69a878 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:43.589807 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c50db9-2b8c-458b-a8a0-0334ba69a878 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:43.589967 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c50db9-2b8c-458b-a8a0-0334ba69a878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:43.660403 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: INFO dbcounter [None req-31c50db9-2b8c-458b-a8a0-0334ba69a878 None None] Registered counter for database keystone Jun 23 09:02:43.681767 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG oslo_db.sqlalchemy.engines [None req-31c50db9-2b8c-458b-a8a0-0334ba69a878 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80654) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 09:02:43.688628 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writer thread running {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 09:02:43.866334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG passlib.handlers.bcrypt [None req-31c50db9-2b8c-458b-a8a0-0334ba69a878 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80654) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 09:02:43.866672 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG passlib.handlers.bcrypt [None req-31c50db9-2b8c-458b-a8a0-0334ba69a878 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80654) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 09:02:43.920813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-31c50db9-2b8c-458b-a8a0-0334ba69a878 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:02:43.922420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-31c50db9-2b8c-458b-a8a0-0334ba69a878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:02:44.043541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:02:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 23 09:02:44.212023 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3b756-c48b-4084-9738-d59d3d3cf166 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:44.287420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: INFO dbcounter [None req-3bc3b756-c48b-4084-9738-d59d3d3cf166 None None] Registered counter for database keystone Jun 23 09:02:44.307574 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG oslo_db.sqlalchemy.engines [None req-3bc3b756-c48b-4084-9738-d59d3d3cf166 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80655) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 09:02:44.313890 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writer thread running {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 09:02:44.441638 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3b756-c48b-4084-9738-d59d3d3cf166 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:44.445872 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b8db6-4cc6-4b1a-8ec2-a6ca61ca9f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:44.446575 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b8db6-4cc6-4b1a-8ec2-a6ca61ca9f4d admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:44.446896 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b8db6-4cc6-4b1a-8ec2-a6ca61ca9f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:44.447197 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b8db6-4cc6-4b1a-8ec2-a6ca61ca9f4d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:44.455842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b8db6-4cc6-4b1a-8ec2-a6ca61ca9f4d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:44.511697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: WARNING keystone.common.rbac_enforcer.enforcer [None req-490b8db6-4cc6-4b1a-8ec2-a6ca61ca9f4d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 09:02:44.512047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b8db6-4cc6-4b1a-8ec2-a6ca61ca9f4d admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:44.523750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Thu Jun 23 09:02:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 09:02:44.529701 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c50db9-2b8c-458b-a8a0-0334ba69a878 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:44.544762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c50db9-2b8c-458b-a8a0-0334ba69a878 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:44.549116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b782f-daff-4e38-9e3a-9bef4b87905e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:44.550235 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b782f-daff-4e38-9e3a-9bef4b87905e admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:44.550524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b782f-daff-4e38-9e3a-9bef4b87905e admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:44.550799 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b782f-daff-4e38-9e3a-9bef4b87905e admin admin] PATH_INFO: `/v3/projects/e5fa9b7cbdd44d0c8b5ffa0c15d81646` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:44.554876 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b782f-daff-4e38-9e3a-9bef4b87905e admin admin] RBAC: Authorizing `identity:get_project(project_id=e5fa9b7cbdd44d0c8b5ffa0c15d81646)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:44.605335 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ae1b782f-daff-4e38-9e3a-9bef4b87905e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 09:02:44.605865 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b782f-daff-4e38-9e3a-9bef4b87905e admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:44.610291 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Thu Jun 23 09:02:44 2022] GET /identity/v3/projects/e5fa9b7cbdd44d0c8b5ffa0c15d81646 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:02:46.602161 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2ff5e-c811-4a22-9f2a-61e716da0ad5 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:46.602668 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2ff5e-c811-4a22-9f2a-61e716da0ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:46.602754 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2ff5e-c811-4a22-9f2a-61e716da0ad5 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:46.606949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:02:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:02:46.612165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf928b4-881d-4e70-8cfc-47aa47de5044 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:46.612560 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf928b4-881d-4e70-8cfc-47aa47de5044 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:46.612742 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf928b4-881d-4e70-8cfc-47aa47de5044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:46.631179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-eaf928b4-881d-4e70-8cfc-47aa47de5044 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:02:46.632717 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-eaf928b4-881d-4e70-8cfc-47aa47de5044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:02:46.641039 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:02:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 23 09:02:46.666182 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2ff5e-c811-4a22-9f2a-61e716da0ad5 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:46.678119 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2ff5e-c811-4a22-9f2a-61e716da0ad5 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:46.682250 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60dbe35-e9b1-4514-922a-14d77cf63162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:46.682938 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dbe35-e9b1-4514-922a-14d77cf63162 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:46.683351 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dbe35-e9b1-4514-922a-14d77cf63162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:46.683627 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dbe35-e9b1-4514-922a-14d77cf63162 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:46.691339 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60dbe35-e9b1-4514-922a-14d77cf63162 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:46.693117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60dbe35-e9b1-4514-922a-14d77cf63162 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:46.702341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Thu Jun 23 09:02:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 09:02:46.709017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf928b4-881d-4e70-8cfc-47aa47de5044 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:46.718566 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf928b4-881d-4e70-8cfc-47aa47de5044 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:46.723069 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1eb68-90e0-40df-a64a-8a32729e13b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:46.723916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1eb68-90e0-40df-a64a-8a32729e13b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:46.724201 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1eb68-90e0-40df-a64a-8a32729e13b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:46.724477 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1eb68-90e0-40df-a64a-8a32729e13b2 admin admin] PATH_INFO: `/v3/users/261a7891b0d244a3a8e71c73d6ba80b6` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:46.725879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1eb68-90e0-40df-a64a-8a32729e13b2 admin admin] RBAC: Authorizing `identity:get_user(user_id=261a7891b0d244a3a8e71c73d6ba80b6)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:46.727750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1eb68-90e0-40df-a64a-8a32729e13b2 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:46.729322 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Thu Jun 23 09:02:46 2022] GET /identity/v3/users/261a7891b0d244a3a8e71c73d6ba80b6 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:02:48.813396 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61885198-fdbe-4a54-8217-7b272a34666b None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:48.813910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61885198-fdbe-4a54-8217-7b272a34666b None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:48.814000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61885198-fdbe-4a54-8217-7b272a34666b None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:48.815348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:02:48.820378 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc38101-ee4c-4c0f-9395-15d99f299d36 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:48.820802 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc38101-ee4c-4c0f-9395-15d99f299d36 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:48.820977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc38101-ee4c-4c0f-9395-15d99f299d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:48.838072 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-abc38101-ee4c-4c0f-9395-15d99f299d36 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:02:48.839287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-abc38101-ee4c-4c0f-9395-15d99f299d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:02:48.882564 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc205a9-319d-4192-9715-6529bf441510 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:48.883123 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc205a9-319d-4192-9715-6529bf441510 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:48.883643 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc205a9-319d-4192-9715-6529bf441510 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:48.886166 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:02:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:02:48.892641 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eeda67-e3d6-4dd2-a2a5-6e5c0b4cbbe7 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:48.892935 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eeda67-e3d6-4dd2-a2a5-6e5c0b4cbbe7 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:48.893219 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eeda67-e3d6-4dd2-a2a5-6e5c0b4cbbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:48.925757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:02:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:02:48.933292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG passlib.handlers.bcrypt [None req-00eeda67-e3d6-4dd2-a2a5-6e5c0b4cbbe7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80655) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 09:02:48.933740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG passlib.handlers.bcrypt [None req-00eeda67-e3d6-4dd2-a2a5-6e5c0b4cbbe7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80655) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 09:02:48.948466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc38101-ee4c-4c0f-9395-15d99f299d36 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:48.973283 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:48.983908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-00eeda67-e3d6-4dd2-a2a5-6e5c0b4cbbe7 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:02:48.985573 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-00eeda67-e3d6-4dd2-a2a5-6e5c0b4cbbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:02:49.054084 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc38101-ee4c-4c0f-9395-15d99f299d36 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:49.086737 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:02:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:02:49.092329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e06ae-95fe-4dca-96f5-c7c9e8cb3476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:49.093128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e06ae-95fe-4dca-96f5-c7c9e8cb3476 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:49.093414 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e06ae-95fe-4dca-96f5-c7c9e8cb3476 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:49.093689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e06ae-95fe-4dca-96f5-c7c9e8cb3476 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:49.104602 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e06ae-95fe-4dca-96f5-c7c9e8cb3476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:02:49.109116 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eeda67-e3d6-4dd2-a2a5-6e5c0b4cbbe7 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:49.116212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-c63e06ae-95fe-4dca-96f5-c7c9e8cb3476 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:02:49.116212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:02:49.116212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:02:49.116212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:02:49.116212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:02:49.116212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:02:49.116212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:02:49.116212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:02:49.116212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:02:49.116212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:02:49.116212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:02:49.116212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:02:49.116212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:02:49.116212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:02:49.116212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:02:49.116212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:02:49.117125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:02:49.117125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:02:49.117125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:02:49.117125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:02:49.117125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:02:49.117125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:02:49.117125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:02:49.117125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:02:49.117125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:02:49.117125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:02:49.117125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:02:49.117125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:02:49.117125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:02:49.117125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:02:49.117125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:02:49.117125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:02:49.118050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:02:49.118050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:02:49.118050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:02:49.118050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:02:49.118050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:02:49.118050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:02:49.118050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:02:49.118050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:02:49.118050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:02:49.118050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:02:49.118050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:02:49.118050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:02:49.118050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:02:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:02:49.123716 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e06ae-95fe-4dca-96f5-c7c9e8cb3476 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:49.209623 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:49.213226 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eeda67-e3d6-4dd2-a2a5-6e5c0b4cbbe7 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:49.215521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:49.227488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e06ae-95fe-4dca-96f5-c7c9e8cb3476 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:49.251245 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e3f5a-2763-4baf-b700-276cb900e7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:49.252175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e3f5a-2763-4baf-b700-276cb900e7ea None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:49.252469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e3f5a-2763-4baf-b700-276cb900e7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:49.252757 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e3f5a-2763-4baf-b700-276cb900e7ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:49.261196 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58a6d2-98ea-4a0f-9d2d-c41e3515a20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:49.263708 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58a6d2-98ea-4a0f-9d2d-c41e3515a20b None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:49.263708 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58a6d2-98ea-4a0f-9d2d-c41e3515a20b None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:49.263708 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58a6d2-98ea-4a0f-9d2d-c41e3515a20b None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:49.264000 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58a6d2-98ea-4a0f-9d2d-c41e3515a20b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:02:49.264289 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58a6d2-98ea-4a0f-9d2d-c41e3515a20b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:49.265726 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58a6d2-98ea-4a0f-9d2d-c41e3515a20b None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:49.277584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:02:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:02:49.283552 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58a6d2-98ea-4a0f-9d2d-c41e3515a20b None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:49.321703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e3f5a-2763-4baf-b700-276cb900e7ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:49.323270 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e3f5a-2763-4baf-b700-276cb900e7ea None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:49.385079 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58a6d2-98ea-4a0f-9d2d-c41e3515a20b None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:49.418421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99889ffc-313f-409b-bfb1-7cba38a11820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:49.418588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 8/16] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:02:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 09:02:49.419904 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99889ffc-313f-409b-bfb1-7cba38a11820 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:49.420273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99889ffc-313f-409b-bfb1-7cba38a11820 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:49.420565 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99889ffc-313f-409b-bfb1-7cba38a11820 None admin] PATH_INFO: `/v3/users/261a7891b0d244a3a8e71c73d6ba80b6` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:49.421819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99889ffc-313f-409b-bfb1-7cba38a11820 None admin] RBAC: Authorizing `identity:get_user(user_id=261a7891b0d244a3a8e71c73d6ba80b6)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:49.423324 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99889ffc-313f-409b-bfb1-7cba38a11820 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:49.424722 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e3f5a-2763-4baf-b700-276cb900e7ea None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:49.429189 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 9/17] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:02:49 2022] GET /identity/v3/users/261a7891b0d244a3a8e71c73d6ba80b6 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:02:49.429350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:49.430121 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99889ffc-313f-409b-bfb1-7cba38a11820 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:49.444685 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:49.522397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e3f5a-2763-4baf-b700-276cb900e7ea None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:49.526659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99889ffc-313f-409b-bfb1-7cba38a11820 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:49.555937 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2926175-9364-4e4c-97ca-c051d3d8fb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:49.556739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2926175-9364-4e4c-97ca-c051d3d8fb93 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:49.557030 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2926175-9364-4e4c-97ca-c051d3d8fb93 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:49.557316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2926175-9364-4e4c-97ca-c051d3d8fb93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:49.560681 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba27b7-b395-4d0e-adde-db472c9641e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:49.561471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba27b7-b395-4d0e-adde-db472c9641e9 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:49.561758 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba27b7-b395-4d0e-adde-db472c9641e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:49.562034 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba27b7-b395-4d0e-adde-db472c9641e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:49.563236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba27b7-b395-4d0e-adde-db472c9641e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:49.564683 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba27b7-b395-4d0e-adde-db472c9641e9 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:49.566283 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 10/18] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:02:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:02:49.571927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba27b7-b395-4d0e-adde-db472c9641e9 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:49.577534 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2926175-9364-4e4c-97ca-c051d3d8fb93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:49.579052 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2926175-9364-4e4c-97ca-c051d3d8fb93 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:49.599914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-a2926175-9364-4e4c-97ca-c051d3d8fb93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:02:49.599914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:02:49.599914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:02:49.599914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:02:49.599914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:02:49.599914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:02:49.599914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:02:49.599914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:02:49.599914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:02:49.599914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:02:49.599914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:02:49.599914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:02:49.599914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:02:49.599914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:02:49.599914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:02:49.600992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:02:49.600992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:02:49.600992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:02:49.600992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:02:49.600992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:02:49.600992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:02:49.600992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:02:49.600992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:02:49.600992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:02:49.600992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:02:49.600992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:02:49.600992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:02:49.600992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:02:49.600992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:02:49.600992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:02:49.600992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:02:49.600992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:02:49.602024 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:02:49.602024 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:02:49.602024 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:02:49.602024 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:02:49.602024 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 9/19] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:02:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:02:49.607371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2926175-9364-4e4c-97ca-c051d3d8fb93 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:49.654799 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:49.663088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:49.673008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba27b7-b395-4d0e-adde-db472c9641e9 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:49.704857 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2926175-9364-4e4c-97ca-c051d3d8fb93 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:49.707333 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a2717-17aa-4d3b-b33c-0276a1c665d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:49.707990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a2717-17aa-4d3b-b33c-0276a1c665d3 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:49.708320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a2717-17aa-4d3b-b33c-0276a1c665d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:49.708579 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a2717-17aa-4d3b-b33c-0276a1c665d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:49.709408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a2717-17aa-4d3b-b33c-0276a1c665d3 None admin] RBAC: Adding query filter params (role.id=9cd080b50c6f4ee8a54bf9ef1655bd83, scope.domain.id=default, user.id=261a7891b0d244a3a8e71c73d6ba80b6) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:02:49.709772 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a2717-17aa-4d3b-b33c-0276a1c665d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:49.711243 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a2717-17aa-4d3b-b33c-0276a1c665d3 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:49.717901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 11/20] 10.222.0.52 () {56 vars in 1322 bytes} [Thu Jun 23 09:02:49 2022] GET /identity/v3/role_assignments?user.id=261a7891b0d244a3a8e71c73d6ba80b6&scope.domain.id=default&role.id=9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:02:49.740635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931698b-30e2-4b6c-84a6-a6ecfdefbcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:49.741331 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931698b-30e2-4b6c-84a6-a6ecfdefbcea None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:49.741621 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931698b-30e2-4b6c-84a6-a6ecfdefbcea None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:49.741903 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931698b-30e2-4b6c-84a6-a6ecfdefbcea None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:49.742665 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3931698b-30e2-4b6c-84a6-a6ecfdefbcea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:02:49.743014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3931698b-30e2-4b6c-84a6-a6ecfdefbcea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:49.744493 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3931698b-30e2-4b6c-84a6-a6ecfdefbcea None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:49.764411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 10/21] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:02:52.019719 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07e7a2-bc18-482c-a99d-e060d0187491 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:52.020735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07e7a2-bc18-482c-a99d-e060d0187491 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:52.021246 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07e7a2-bc18-482c-a99d-e060d0187491 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:52.023088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 12/22] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:02:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:02:52.028653 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191e5bd-e3b5-41e6-a141-9b4c1a956e2d None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:52.029479 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191e5bd-e3b5-41e6-a141-9b4c1a956e2d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:52.029956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191e5bd-e3b5-41e6-a141-9b4c1a956e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:52.052148 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-e191e5bd-e3b5-41e6-a141-9b4c1a956e2d None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:02:52.053689 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e191e5bd-e3b5-41e6-a141-9b4c1a956e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:02:52.080727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea2b0d-09eb-4e1c-8f1b-cabc8aaba6a1 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:52.081330 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea2b0d-09eb-4e1c-8f1b-cabc8aaba6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:52.082054 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea2b0d-09eb-4e1c-8f1b-cabc8aaba6a1 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:52.083653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 13/23] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:02:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:02:52.098748 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1e5c9-b5e2-43fc-9c14-a6c5d3642e08 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:52.099067 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1e5c9-b5e2-43fc-9c14-a6c5d3642e08 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:52.099348 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1e5c9-b5e2-43fc-9c14-a6c5d3642e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:52.118629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-50f1e5c9-b5e2-43fc-9c14-a6c5d3642e08 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:02:52.120037 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-50f1e5c9-b5e2-43fc-9c14-a6c5d3642e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:02:52.162011 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 11/24] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:02:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:02:52.207455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191e5bd-e3b5-41e6-a141-9b4c1a956e2d None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:52.247707 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 14/25] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:02:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:02:52.281553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:52.293175 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1e5c9-b5e2-43fc-9c14-a6c5d3642e08 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:52.386176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191e5bd-e3b5-41e6-a141-9b4c1a956e2d None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:52.436509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:52.453607 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07373a8-0cc0-414a-afcd-812107df5997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:52.454418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07373a8-0cc0-414a-afcd-812107df5997 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:52.454703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07373a8-0cc0-414a-afcd-812107df5997 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:52.455000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07373a8-0cc0-414a-afcd-812107df5997 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:52.477964 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07373a8-0cc0-414a-afcd-812107df5997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:52.480631 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07373a8-0cc0-414a-afcd-812107df5997 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:52.486822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1e5c9-b5e2-43fc-9c14-a6c5d3642e08 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:52.527477 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b6425-7b17-460c-a005-5f699a205ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:52.528719 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b6425-7b17-460c-a005-5f699a205ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:52.529203 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b6425-7b17-460c-a005-5f699a205ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:52.529671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b6425-7b17-460c-a005-5f699a205ef9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:52.535448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-b07373a8-0cc0-414a-afcd-812107df5997 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:02:52.535448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:02:52.535448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:02:52.535448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:02:52.535448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:02:52.535448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:02:52.535448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:02:52.535448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:02:52.535448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:02:52.535448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:02:52.535448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:02:52.535448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:02:52.535448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:02:52.535448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:02:52.535448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:02:52.536280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:02:52.536280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:02:52.536280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:02:52.536280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:02:52.536280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:02:52.536280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:02:52.536280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:02:52.536280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:02:52.536280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:02:52.536280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:02:52.536280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:02:52.536280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:02:52.536280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:02:52.536280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:02:52.536280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:02:52.536280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:02:52.536280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:02:52.537188 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:02:52.537188 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:02:52.537188 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:02:52.537188 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:02:52.537188 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 12/26] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:02:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:02:52.538512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63a75a-1644-49e2-a2f4-a7481c6d6739 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:52.538770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63a75a-1644-49e2-a2f4-a7481c6d6739 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:52.539087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63a75a-1644-49e2-a2f4-a7481c6d6739 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:52.544392 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b6425-7b17-460c-a005-5f699a205ef9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:02:52.545053 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 13/27] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:02:52.545871 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63a75a-1644-49e2-a2f4-a7481c6d6739 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:52.552974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-f80b6425-7b17-460c-a005-5f699a205ef9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:02:52.552974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:02:52.552974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:02:52.552974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:02:52.552974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:02:52.552974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:02:52.552974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:02:52.552974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:02:52.552974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:02:52.552974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:02:52.552974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:02:52.552974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:02:52.552974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:02:52.552974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:02:52.552974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:02:52.552974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:02:52.553855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:02:52.553855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:02:52.553855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:02:52.553855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:02:52.553855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:02:52.553855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:02:52.553855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:02:52.553855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:02:52.553855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:02:52.553855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:02:52.553855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:02:52.553855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:02:52.553855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:02:52.553855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:02:52.553855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:02:52.553855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:02:52.554671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:02:52.554671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:02:52.554671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:02:52.554671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:02:52.554671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:02:52.554671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:02:52.554671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:02:52.554671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:02:52.554671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:02:52.554671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:02:52.554671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:02:52.554671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:02:52.557400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 15/28] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:02:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:02:52.559983 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671afae-8f98-45a3-9911-2b1918ce5e5c None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:52.560545 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671afae-8f98-45a3-9911-2b1918ce5e5c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:52.561023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671afae-8f98-45a3-9911-2b1918ce5e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:52.591949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-2671afae-8f98-45a3-9911-2b1918ce5e5c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:02:52.593179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-2671afae-8f98-45a3-9911-2b1918ce5e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:02:52.710546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 16/29] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:02:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:02:52.716102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671afae-8f98-45a3-9911-2b1918ce5e5c None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:52.726612 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:52.736587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63a75a-1644-49e2-a2f4-a7481c6d6739 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:52.778220 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd9c85-b196-4073-bb09-1898e88d0c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:52.778984 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd9c85-b196-4073-bb09-1898e88d0c95 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:52.779278 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd9c85-b196-4073-bb09-1898e88d0c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:52.779618 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd9c85-b196-4073-bb09-1898e88d0c95 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:52.780377 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd9c85-b196-4073-bb09-1898e88d0c95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:02:52.780820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd9c85-b196-4073-bb09-1898e88d0c95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:52.782416 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd9c85-b196-4073-bb09-1898e88d0c95 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:52.796389 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 14/30] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:02:52.797497 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd9c85-b196-4073-bb09-1898e88d0c95 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:52.805480 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:52.844761 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671afae-8f98-45a3-9911-2b1918ce5e5c None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:52.881005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5398d-6c76-4c51-9c04-a733ba6f7bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:52.881924 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5398d-6c76-4c51-9c04-a733ba6f7bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:52.882360 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5398d-6c76-4c51-9c04-a733ba6f7bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:52.882855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5398d-6c76-4c51-9c04-a733ba6f7bd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:52.883837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5398d-6c76-4c51-9c04-a733ba6f7bd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:02:52.884357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5398d-6c76-4c51-9c04-a733ba6f7bd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:52.885972 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5398d-6c76-4c51-9c04-a733ba6f7bd3 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:52.899041 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 17/31] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:02:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:02:52.900162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5398d-6c76-4c51-9c04-a733ba6f7bd3 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:52.906295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd9c85-b196-4073-bb09-1898e88d0c95 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:52.946940 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1584cb-6e56-4205-9457-f67b301e4677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:52.946940 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1584cb-6e56-4205-9457-f67b301e4677 None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:52.946940 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1584cb-6e56-4205-9457-f67b301e4677 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:52.946940 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1584cb-6e56-4205-9457-f67b301e4677 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:52.947360 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1584cb-6e56-4205-9457-f67b301e4677 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:52.947360 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1584cb-6e56-4205-9457-f67b301e4677 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:53.049683 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5398d-6c76-4c51-9c04-a733ba6f7bd3 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:53.078550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:53.095670 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f6e12-f8e5-4514-9264-819c2eaa1670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:53.100057 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f6e12-f8e5-4514-9264-819c2eaa1670 None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:53.100057 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f6e12-f8e5-4514-9264-819c2eaa1670 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:53.100057 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f6e12-f8e5-4514-9264-819c2eaa1670 None admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:53.100057 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f6e12-f8e5-4514-9264-819c2eaa1670 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:53.101672 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f6e12-f8e5-4514-9264-819c2eaa1670 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:53.198953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 15/32] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Jun 23 09:02:52 2022] POST /identity/v3/roles => generated 219 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:02:53.198953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1584cb-6e56-4205-9457-f67b301e4677 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:53.250339 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:53.399467 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 18/33] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jun 23 09:02:52 2022] POST /identity/v3/projects => generated 301 bytes in 500 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:02:53.401104 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea3db1-3de0-4e3f-a9e4-e79725f262a7 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:53.401560 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea3db1-3de0-4e3f-a9e4-e79725f262a7 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:53.401989 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea3db1-3de0-4e3f-a9e4-e79725f262a7 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:53.404515 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 19/34] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:02:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:02:53.413982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a8db0-893a-452c-8ad1-0b2dbc929a73 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:53.414631 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a8db0-893a-452c-8ad1-0b2dbc929a73 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:53.415126 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a8db0-893a-452c-8ad1-0b2dbc929a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:53.433144 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1584cb-6e56-4205-9457-f67b301e4677 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:53.446930 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-a13a8db0-893a-452c-8ad1-0b2dbc929a73 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:02:53.449726 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a13a8db0-893a-452c-8ad1-0b2dbc929a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:02:53.507378 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20583ff-f57d-46e7-b3a6-2c532542846b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:53.508576 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20583ff-f57d-46e7-b3a6-2c532542846b None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:53.509804 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20583ff-f57d-46e7-b3a6-2c532542846b None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:53.510898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20583ff-f57d-46e7-b3a6-2c532542846b None admin] PATH_INFO: `/v3/users/261a7891b0d244a3a8e71c73d6ba80b6` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:53.515444 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20583ff-f57d-46e7-b3a6-2c532542846b None admin] RBAC: Authorizing `identity:get_user(user_id=261a7891b0d244a3a8e71c73d6ba80b6)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:53.520785 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20583ff-f57d-46e7-b3a6-2c532542846b None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:53.530461 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 16/35] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:02:53 2022] GET /identity/v3/users/261a7891b0d244a3a8e71c73d6ba80b6 => generated 247 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:02:53.536948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20583ff-f57d-46e7-b3a6-2c532542846b None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:53.596189 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 20/36] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:02:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:02:53.632774 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a8db0-893a-452c-8ad1-0b2dbc929a73 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:53.664275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:53.682031 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20583ff-f57d-46e7-b3a6-2c532542846b None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:53.729748 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8457153-6530-4d95-8628-59b722a41630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:53.730564 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8457153-6530-4d95-8628-59b722a41630 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:53.730857 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8457153-6530-4d95-8628-59b722a41630 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:53.731165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8457153-6530-4d95-8628-59b722a41630 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:53.732566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8457153-6530-4d95-8628-59b722a41630 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:53.734040 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8457153-6530-4d95-8628-59b722a41630 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:53.735884 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 17/37] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:02:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:02:53.741898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bd1f8-94e8-4bc5-b1bb-a1700851184e None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:53.741898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bd1f8-94e8-4bc5-b1bb-a1700851184e None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:53.741898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bd1f8-94e8-4bc5-b1bb-a1700851184e None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:53.741898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 18/38] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:02:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:02:53.743101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09bd1f8-94e8-4bc5-b1bb-a1700851184e None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:53.767087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a8db0-893a-452c-8ad1-0b2dbc929a73 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:53.777618 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:53.806266 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6be01-922e-4c95-a6b9-6ef093234ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:53.806997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6be01-922e-4c95-a6b9-6ef093234ae5 None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:53.807583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6be01-922e-4c95-a6b9-6ef093234ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:53.807867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6be01-922e-4c95-a6b9-6ef093234ae5 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:53.808636 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6be01-922e-4c95-a6b9-6ef093234ae5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:53.810704 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6be01-922e-4c95-a6b9-6ef093234ae5 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:53.852425 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09bd1f8-94e8-4bc5-b1bb-a1700851184e None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:53.885245 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17b52b-6bab-4ee9-a8c3-f09c97b85279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:53.886114 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17b52b-6bab-4ee9-a8c3-f09c97b85279 None admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:53.886404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17b52b-6bab-4ee9-a8c3-f09c97b85279 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:53.886686 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17b52b-6bab-4ee9-a8c3-f09c97b85279 None admin] PATH_INFO: `/v3/domains/default/users/261a7891b0d244a3a8e71c73d6ba80b6/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:53.889303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17b52b-6bab-4ee9-a8c3-f09c97b85279 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=261a7891b0d244a3a8e71c73d6ba80b6, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:53.890738 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17b52b-6bab-4ee9-a8c3-f09c97b85279 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:53.901420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 21/39] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Jun 23 09:02:53 2022] POST /identity/v3/roles => generated 225 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:02:53.902865 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5e5c1-e3f3-4de6-a0d1-c632f4535b36 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:53.922741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5e5c1-e3f3-4de6-a0d1-c632f4535b36 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:53.922741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5e5c1-e3f3-4de6-a0d1-c632f4535b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:53.927676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-f9b5e5c1-e3f3-4de6-a0d1-c632f4535b36 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:02:53.928938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-f9b5e5c1-e3f3-4de6-a0d1-c632f4535b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:02:54.002756 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 22/40] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:02:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:02:54.004154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da7cd6-4d24-427b-bf44-a9a8ec15ffcf None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:54.004455 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da7cd6-4d24-427b-bf44-a9a8ec15ffcf None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:54.004876 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da7cd6-4d24-427b-bf44-a9a8ec15ffcf None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:54.006465 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 23/41] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:02:54.011982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71ae32-1a8c-4de9-9ca6-b73e40817f39 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:54.012282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71ae32-1a8c-4de9-9ca6-b73e40817f39 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:54.012568 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71ae32-1a8c-4de9-9ca6-b73e40817f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:54.031096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-9f71ae32-1a8c-4de9-9ca6-b73e40817f39 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:02:54.032329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-9f71ae32-1a8c-4de9-9ca6-b73e40817f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:02:54.050023 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 19/42] 10.222.0.52 () {58 vars in 1361 bytes} [Thu Jun 23 09:02:53 2022] PUT /identity/v3/domains/default/users/261a7891b0d244a3a8e71c73d6ba80b6/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:02:54.050023 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17b52b-6bab-4ee9-a8c3-f09c97b85279 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:54.085602 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:54.200988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17b52b-6bab-4ee9-a8c3-f09c97b85279 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:54.237505 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 24/43] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:02:54.243516 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26fe760-70f7-45ce-98f1-48a26e635899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:54.244204 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fe760-70f7-45ce-98f1-48a26e635899 None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:54.244499 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fe760-70f7-45ce-98f1-48a26e635899 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:54.244786 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fe760-70f7-45ce-98f1-48a26e635899 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:54.245558 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26fe760-70f7-45ce-98f1-48a26e635899 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:54.247006 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26fe760-70f7-45ce-98f1-48a26e635899 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:54.260604 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71ae32-1a8c-4de9-9ca6-b73e40817f39 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:54.289092 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:54.359477 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71ae32-1a8c-4de9-9ca6-b73e40817f39 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:54.381463 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 20/44] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Jun 23 09:02:54 2022] POST /identity/v3/roles => generated 223 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:02:54.394056 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeecdc1e-0487-4b35-93dc-469272c08957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:54.394898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeecdc1e-0487-4b35-93dc-469272c08957 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:54.395240 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeecdc1e-0487-4b35-93dc-469272c08957 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:54.395524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeecdc1e-0487-4b35-93dc-469272c08957 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:54.396935 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeecdc1e-0487-4b35-93dc-469272c08957 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:54.398428 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeecdc1e-0487-4b35-93dc-469272c08957 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:54.400479 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 25/45] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:02:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:02:54.406485 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26fe760-70f7-45ce-98f1-48a26e635899 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:54.490544 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:54.518460 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26fe760-70f7-45ce-98f1-48a26e635899 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:54.555501 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f67fcf-588d-4851-b1d2-fb42d56fc2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:54.556179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f67fcf-588d-4851-b1d2-fb42d56fc2b2 None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:54.556466 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f67fcf-588d-4851-b1d2-fb42d56fc2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:54.556745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f67fcf-588d-4851-b1d2-fb42d56fc2b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:54.557477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f67fcf-588d-4851-b1d2-fb42d56fc2b2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:54.558963 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f67fcf-588d-4851-b1d2-fb42d56fc2b2 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:54.691291 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 21/46] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jun 23 09:02:54 2022] POST /identity/v3/projects => generated 312 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:02:56.138513 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14110f5b-a521-466c-ba25-3f96f3ca679e None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:56.139057 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14110f5b-a521-466c-ba25-3f96f3ca679e None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:56.139170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14110f5b-a521-466c-ba25-3f96f3ca679e None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:56.140480 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 26/47] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:02:56.145661 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646726d-4a55-4fd1-8f12-c157ee227297 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:56.146038 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646726d-4a55-4fd1-8f12-c157ee227297 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:56.146366 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646726d-4a55-4fd1-8f12-c157ee227297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:56.166329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-e646726d-4a55-4fd1-8f12-c157ee227297 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:02:56.167917 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e646726d-4a55-4fd1-8f12-c157ee227297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:02:56.234767 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 22/48] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:02:56.257796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14110f5b-a521-466c-ba25-3f96f3ca679e None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:56.379730 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:56.397767 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14110f5b-a521-466c-ba25-3f96f3ca679e None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:56.427940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e55b97-861e-44cd-b2dd-32d1e1eb9cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:56.428743 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e55b97-861e-44cd-b2dd-32d1e1eb9cbd None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:56.429032 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e55b97-861e-44cd-b2dd-32d1e1eb9cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:56.429313 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e55b97-861e-44cd-b2dd-32d1e1eb9cbd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:56.439921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e55b97-861e-44cd-b2dd-32d1e1eb9cbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:02:56.447442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-d0e55b97-861e-44cd-b2dd-32d1e1eb9cbd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:02:56.447442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:02:56.447442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:02:56.447442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:02:56.447442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:02:56.447442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:02:56.447442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:02:56.447442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:02:56.447442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:02:56.447442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:02:56.447442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:02:56.447442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:02:56.447442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:02:56.447442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:02:56.447442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:02:56.447442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:02:56.448471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:02:56.448471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:02:56.448471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:02:56.448471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:02:56.448471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:02:56.448471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:02:56.448471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:02:56.448471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:02:56.448471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:02:56.448471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:02:56.448471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:02:56.448471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:02:56.448471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:02:56.448471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:02:56.448471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:02:56.448471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:02:56.449314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:02:56.449314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:02:56.449314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:02:56.449314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:02:56.449314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:02:56.449314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:02:56.449314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:02:56.449314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:02:56.449314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:02:56.449314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:02:56.449314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:02:56.449314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:02:56.449314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 27/49] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:02:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:02:56.454490 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646726d-4a55-4fd1-8f12-c157ee227297 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:56.510200 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:56.558277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646726d-4a55-4fd1-8f12-c157ee227297 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:56.592364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d435e-d206-4a10-86cd-767374f89b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:56.593041 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d435e-d206-4a10-86cd-767374f89b29 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:56.593329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d435e-d206-4a10-86cd-767374f89b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:56.593607 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d435e-d206-4a10-86cd-767374f89b29 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:56.594367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d435e-d206-4a10-86cd-767374f89b29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:02:56.594710 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d435e-d206-4a10-86cd-767374f89b29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:56.596115 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d435e-d206-4a10-86cd-767374f89b29 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:56.610868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 23/50] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:02:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:02:56.617078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e55b97-861e-44cd-b2dd-32d1e1eb9cbd None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:56.720022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e55b97-861e-44cd-b2dd-32d1e1eb9cbd None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:56.736156 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e83ced-a0dc-4b57-b96a-d04d5e967852 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:56.736452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e83ced-a0dc-4b57-b96a-d04d5e967852 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:56.736738 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e83ced-a0dc-4b57-b96a-d04d5e967852 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:56.738126 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 24/51] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:02:56.743006 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd8703-f339-4ce6-a4a0-23de3c679bd8 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:56.743306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd8703-f339-4ce6-a4a0-23de3c679bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:56.743596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd8703-f339-4ce6-a4a0-23de3c679bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:56.753003 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36dcfdf-41e5-400a-b35c-3eaf2d6a5823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:56.753883 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36dcfdf-41e5-400a-b35c-3eaf2d6a5823 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:56.754175 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36dcfdf-41e5-400a-b35c-3eaf2d6a5823 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:56.754457 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36dcfdf-41e5-400a-b35c-3eaf2d6a5823 None admin] PATH_INFO: `/v3/users/261a7891b0d244a3a8e71c73d6ba80b6` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:56.755688 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36dcfdf-41e5-400a-b35c-3eaf2d6a5823 None admin] RBAC: Authorizing `identity:get_user(user_id=261a7891b0d244a3a8e71c73d6ba80b6)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:56.757187 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36dcfdf-41e5-400a-b35c-3eaf2d6a5823 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:56.758925 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 28/52] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:02:56 2022] GET /identity/v3/users/261a7891b0d244a3a8e71c73d6ba80b6 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:02:56.765219 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36dcfdf-41e5-400a-b35c-3eaf2d6a5823 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:56.792493 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:56.864981 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-8bcd8703-f339-4ce6-a4a0-23de3c679bd8 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:02:56.866205 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-8bcd8703-f339-4ce6-a4a0-23de3c679bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:02:56.870033 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36dcfdf-41e5-400a-b35c-3eaf2d6a5823 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:56.904508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd3ee6-a2f5-45e9-b787-81aa8bbcb4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:56.905308 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd3ee6-a2f5-45e9-b787-81aa8bbcb4fb None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:56.905606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd3ee6-a2f5-45e9-b787-81aa8bbcb4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:56.905887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd3ee6-a2f5-45e9-b787-81aa8bbcb4fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:56.907125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd3ee6-a2f5-45e9-b787-81aa8bbcb4fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:56.908608 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd3ee6-a2f5-45e9-b787-81aa8bbcb4fb None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:56.910437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 29/53] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:02:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:02:56.916079 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd3ee6-a2f5-45e9-b787-81aa8bbcb4fb None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:56.935697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 25/54] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:02:56.963292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd8703-f339-4ce6-a4a0-23de3c679bd8 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:57.012632 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:57.023509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:57.032071 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd3ee6-a2f5-45e9-b787-81aa8bbcb4fb None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:57.064320 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd8703-f339-4ce6-a4a0-23de3c679bd8 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:57.071292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ade36-030c-4087-aa1c-f37bad64cbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:57.072164 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ade36-030c-4087-aa1c-f37bad64cbb2 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:57.072503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ade36-030c-4087-aa1c-f37bad64cbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:57.072944 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ade36-030c-4087-aa1c-f37bad64cbb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:57.074134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ade36-030c-4087-aa1c-f37bad64cbb2 None admin] RBAC: Adding query filter params (role.id=9cd080b50c6f4ee8a54bf9ef1655bd83, scope.domain.id=default, user.id=261a7891b0d244a3a8e71c73d6ba80b6) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:02:57.074455 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ade36-030c-4087-aa1c-f37bad64cbb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:57.076495 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ade36-030c-4087-aa1c-f37bad64cbb2 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:57.085759 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 30/55] 10.222.0.52 () {56 vars in 1322 bytes} [Thu Jun 23 09:02:56 2022] GET /identity/v3/role_assignments?user.id=261a7891b0d244a3a8e71c73d6ba80b6&scope.domain.id=default&role.id=9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 530 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:02:57.100346 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a22e8-3106-4c5e-81fb-2f02ec5c55d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:57.101184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a22e8-3106-4c5e-81fb-2f02ec5c55d2 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:57.101478 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a22e8-3106-4c5e-81fb-2f02ec5c55d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:57.101760 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a22e8-3106-4c5e-81fb-2f02ec5c55d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:57.103091 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a22e8-3106-4c5e-81fb-2f02ec5c55d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:57.106949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a22e8-3106-4c5e-81fb-2f02ec5c55d2 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:57.106949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 26/56] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:02:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:02:57.112417 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ade36-030c-4087-aa1c-f37bad64cbb2 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:57.220992 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ade36-030c-4087-aa1c-f37bad64cbb2 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:57.255990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658109cf-aea8-4adb-ab6e-dddddc8291e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:57.256779 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658109cf-aea8-4adb-ab6e-dddddc8291e8 None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:57.257072 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658109cf-aea8-4adb-ab6e-dddddc8291e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:57.257356 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658109cf-aea8-4adb-ab6e-dddddc8291e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:57.258102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658109cf-aea8-4adb-ab6e-dddddc8291e8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:57.259611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658109cf-aea8-4adb-ab6e-dddddc8291e8 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:57.399216 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 31/57] 10.222.0.52 () {60 vars in 1140 bytes} [Thu Jun 23 09:02:57 2022] POST /identity/v3/projects => generated 298 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:02:59.429119 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fc5c3-0910-4660-b919-8293dccc948e None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:59.429653 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fc5c3-0910-4660-b919-8293dccc948e None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:59.429653 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fc5c3-0910-4660-b919-8293dccc948e None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:59.431026 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 27/58] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:02:59.436662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92757a41-7066-4194-9293-9e607137bf98 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:59.437057 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92757a41-7066-4194-9293-9e607137bf98 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:59.437223 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92757a41-7066-4194-9293-9e607137bf98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:59.453558 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-92757a41-7066-4194-9293-9e607137bf98 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:02:59.454752 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-92757a41-7066-4194-9293-9e607137bf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:02:59.456704 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:59.519654 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 32/59] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:02:59.543147 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377fc5c3-0910-4660-b919-8293dccc948e None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:59.642629 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377fc5c3-0910-4660-b919-8293dccc948e None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:59.658803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:59.675855 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e2298-5f57-4587-8ea5-9bd4a71167a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:59.677036 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e2298-5f57-4587-8ea5-9bd4a71167a3 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:59.677036 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e2298-5f57-4587-8ea5-9bd4a71167a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:59.677298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e2298-5f57-4587-8ea5-9bd4a71167a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:59.678422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e2298-5f57-4587-8ea5-9bd4a71167a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:59.679967 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e2298-5f57-4587-8ea5-9bd4a71167a3 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:59.681531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 28/60] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:02:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:02:59.686818 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92757a41-7066-4194-9293-9e607137bf98 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:02:59.784543 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92757a41-7066-4194-9293-9e607137bf98 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:02:59.816729 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd4f2f-c687-4192-b185-37a7c2b2ce53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:02:59.817392 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd4f2f-c687-4192-b185-37a7c2b2ce53 None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:02:59.817676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd4f2f-c687-4192-b185-37a7c2b2ce53 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:02:59.817952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd4f2f-c687-4192-b185-37a7c2b2ce53 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:02:59.818660 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd4f2f-c687-4192-b185-37a7c2b2ce53 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:02:59.820137 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd4f2f-c687-4192-b185-37a7c2b2ce53 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:02:59.832343 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:02:59.949363 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 33/61] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jun 23 09:02:59 2022] POST /identity/v3/users => generated 275 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:03:03.247857 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b09e42-3bc4-4363-bcaf-f5c9a0a1823a None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:03.248414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b09e42-3bc4-4363-bcaf-f5c9a0a1823a None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:03.248414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b09e42-3bc4-4363-bcaf-f5c9a0a1823a None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:03.249829 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 29/62] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:03.255127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d822a-98f5-49a7-ad35-8a0d66fd5264 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:03.255548 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d822a-98f5-49a7-ad35-8a0d66fd5264 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:03.255737 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d822a-98f5-49a7-ad35-8a0d66fd5264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:03.275508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-9d7d822a-98f5-49a7-ad35-8a0d66fd5264 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:03.276746 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-9d7d822a-98f5-49a7-ad35-8a0d66fd5264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:03.287907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73430ef-6a61-43f0-88e7-33e5beae9193 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:03.288204 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73430ef-6a61-43f0-88e7-33e5beae9193 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:03.288487 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73430ef-6a61-43f0-88e7-33e5beae9193 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:03.290177 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 30/63] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:03.295373 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a147a-321b-40ee-9814-1ab195456351 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:03.295668 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a147a-321b-40ee-9814-1ab195456351 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:03.295953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a147a-321b-40ee-9814-1ab195456351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:03.313566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-130a147a-321b-40ee-9814-1ab195456351 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:03.315048 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-130a147a-321b-40ee-9814-1ab195456351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:03.373521 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 34/64] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:03.378970 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad071c7-1f79-4a23-9196-b0464f3b0809 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:03.378970 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad071c7-1f79-4a23-9196-b0464f3b0809 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:03.378970 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad071c7-1f79-4a23-9196-b0464f3b0809 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:03.379245 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 35/65] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:03.392708 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 31/66] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:03.402345 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad071c7-1f79-4a23-9196-b0464f3b0809 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:03.409955 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae18c94-c220-4152-9556-9ca14db61e85 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:03.410191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae18c94-c220-4152-9556-9ca14db61e85 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:03.410483 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae18c94-c220-4152-9556-9ca14db61e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:03.431685 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-cae18c94-c220-4152-9556-9ca14db61e85 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:03.433238 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-cae18c94-c220-4152-9556-9ca14db61e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:03.507317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 32/67] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:03:03.508744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae18c94-c220-4152-9556-9ca14db61e85 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:03.525103 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad071c7-1f79-4a23-9196-b0464f3b0809 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:03.551242 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:03.563039 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce766ce0-e968-4972-888b-cc1092163bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:03.565690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce766ce0-e968-4972-888b-cc1092163bdb None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:03.565690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce766ce0-e968-4972-888b-cc1092163bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:03.565690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce766ce0-e968-4972-888b-cc1092163bdb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:03.575167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:03.576393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce766ce0-e968-4972-888b-cc1092163bdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:03.584504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-ce766ce0-e968-4972-888b-cc1092163bdb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:03.584504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:03.584504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:03.584504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:03.584504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:03.584504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:03.584504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:03.584504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:03.584504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:03.584504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:03.584504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:03.584504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:03.584504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:03.584504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:03.584504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:03.584504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:03.585509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:03.585509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:03.585509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:03.585509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:03.585509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:03.585509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:03.585509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:03.585509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:03.585509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:03.585509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:03.585509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:03.585509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:03.585509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:03.585509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:03.585509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:03.585509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:03.587108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:03.587108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:03.587108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:03.587108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:03.587108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:03.587108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:03.587108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:03.587108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:03.587108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:03.587108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:03.587108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:03.587108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:03.587108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 36/68] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:03.587108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce766ce0-e968-4972-888b-cc1092163bdb None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:03.647985 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae18c94-c220-4152-9556-9ca14db61e85 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:03.684047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643363b-3d9d-4ae7-a5f6-611a49f58f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:03.685658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643363b-3d9d-4ae7-a5f6-611a49f58f03 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:03.686116 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643363b-3d9d-4ae7-a5f6-611a49f58f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:03.686390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643363b-3d9d-4ae7-a5f6-611a49f58f03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:03.696517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce766ce0-e968-4972-888b-cc1092163bdb None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:03.698712 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643363b-3d9d-4ae7-a5f6-611a49f58f03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:03.708905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-b643363b-3d9d-4ae7-a5f6-611a49f58f03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:03:03.708905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:03.708905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:03.708905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:03.708905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:03.708905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:03.708905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:03.708905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:03.708905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:03.708905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:03.708905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:03.708905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:03.708905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:03.708905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:03.708905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:03.708905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:03.710020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:03.710020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:03.710020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:03.710020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:03.710020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:03.710020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:03.710020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:03.710020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:03.710020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:03.710020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:03.710020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:03.710020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:03.710020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:03.710020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:03.710020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:03.710020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:03.710927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:03.710927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:03.710927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:03.710927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:03.710927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:03.710927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:03.710927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:03.710927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:03.710927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:03.710927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:03.710927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:03:03.710927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:03.712270 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 33/69] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:03:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:03.713095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643363b-3d9d-4ae7-a5f6-611a49f58f03 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:03.735596 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038dc8d9-ea53-4f21-b6b0-629340d59c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:03.736432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dc8d9-ea53-4f21-b6b0-629340d59c45 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:03.736725 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dc8d9-ea53-4f21-b6b0-629340d59c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:03.737185 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dc8d9-ea53-4f21-b6b0-629340d59c45 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:03.748247 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038dc8d9-ea53-4f21-b6b0-629340d59c45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:03.756427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-038dc8d9-ea53-4f21-b6b0-629340d59c45 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:03.756427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:03.756427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:03.756427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:03.756427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:03.756427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:03.756427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:03.756427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:03.756427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:03.756427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:03.756427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:03.756427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:03.756427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:03.756427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:03.756427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:03.756427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:03.757420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:03.757420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:03.757420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:03.757420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:03.757420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:03.757420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:03.757420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:03.757420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:03.757420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:03.757420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:03.757420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:03.757420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:03.757420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:03.757420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:03.757420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:03.757420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:03.758231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:03.758231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:03.758231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:03.758231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:03.758231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:03.758231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:03.758231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:03.758231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:03.758231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:03.758231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:03.758231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:03.758231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:03.758231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 37/70] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:03:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:03.758903 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217957f3-c919-429b-bd18-e539af20c6c8 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:03.759157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217957f3-c919-429b-bd18-e539af20c6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:03.759443 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217957f3-c919-429b-bd18-e539af20c6c8 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:03.760885 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 38/71] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:03.761740 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217957f3-c919-429b-bd18-e539af20c6c8 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:03.809002 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:03.816716 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643363b-3d9d-4ae7-a5f6-611a49f58f03 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:03.819013 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:03.849082 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69907a9-08ec-4bfa-ac74-ba31163031c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:03.849752 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69907a9-08ec-4bfa-ac74-ba31163031c6 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:03.850043 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69907a9-08ec-4bfa-ac74-ba31163031c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:03.850324 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69907a9-08ec-4bfa-ac74-ba31163031c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:03.851109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69907a9-08ec-4bfa-ac74-ba31163031c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:03.851423 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69907a9-08ec-4bfa-ac74-ba31163031c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:03.852800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69907a9-08ec-4bfa-ac74-ba31163031c6 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:03.861170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 34/72] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:03.869058 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c13796-7ff1-48d9-88b3-04a01f2fc4c1 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:03.869058 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c13796-7ff1-48d9-88b3-04a01f2fc4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:03.869058 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c13796-7ff1-48d9-88b3-04a01f2fc4c1 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:03.869058 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 35/73] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:03.869373 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217957f3-c919-429b-bd18-e539af20c6c8 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:03.869455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c13796-7ff1-48d9-88b3-04a01f2fc4c1 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:03.900571 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc9f96-5f01-4938-9dbb-952e7c97229e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:03.901252 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc9f96-5f01-4938-9dbb-952e7c97229e None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:03.901550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc9f96-5f01-4938-9dbb-952e7c97229e None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:03.901837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc9f96-5f01-4938-9dbb-952e7c97229e None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:03.902604 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc9f96-5f01-4938-9dbb-952e7c97229e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:03.902937 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc9f96-5f01-4938-9dbb-952e7c97229e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:03.904438 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc9f96-5f01-4938-9dbb-952e7c97229e None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:03.912932 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 39/74] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:03.914179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3102f4-dd54-4f15-af75-52c445a8bd8f None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:03.914476 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3102f4-dd54-4f15-af75-52c445a8bd8f None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:03.914762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3102f4-dd54-4f15-af75-52c445a8bd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:03.933125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-ba3102f4-dd54-4f15-af75-52c445a8bd8f None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:03.934398 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ba3102f4-dd54-4f15-af75-52c445a8bd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:03.977738 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c13796-7ff1-48d9-88b3-04a01f2fc4c1 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:04.001631 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 40/75] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:04.002472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3102f4-dd54-4f15-af75-52c445a8bd8f None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:04.012855 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a874415-0d0f-4913-bbb1-723137cdedc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:04.013650 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a874415-0d0f-4913-bbb1-723137cdedc2 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:04.013945 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a874415-0d0f-4913-bbb1-723137cdedc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:04.014231 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a874415-0d0f-4913-bbb1-723137cdedc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:04.015018 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a874415-0d0f-4913-bbb1-723137cdedc2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:04.015335 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a874415-0d0f-4913-bbb1-723137cdedc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:04.016947 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a874415-0d0f-4913-bbb1-723137cdedc2 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:04.025559 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 36/76] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:03:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:04.026787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc932a58-ec9b-44a4-b3cb-6235831bf29d None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:04.027185 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc932a58-ec9b-44a4-b3cb-6235831bf29d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:04.027474 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc932a58-ec9b-44a4-b3cb-6235831bf29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:04.040468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:04.043477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-cc932a58-ec9b-44a4-b3cb-6235831bf29d None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:04.045209 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-cc932a58-ec9b-44a4-b3cb-6235831bf29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:04.067835 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:04.105194 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3102f4-dd54-4f15-af75-52c445a8bd8f None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:04.110857 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 37/77] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:04.111711 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc932a58-ec9b-44a4-b3cb-6235831bf29d None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:04.140129 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef1a0f-7dba-4376-a80e-75b0190cfd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:04.141012 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef1a0f-7dba-4376-a80e-75b0190cfd9d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:04.141306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef1a0f-7dba-4376-a80e-75b0190cfd9d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:04.141589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef1a0f-7dba-4376-a80e-75b0190cfd9d None admin] PATH_INFO: `/v3/users/ee9d005c534c4b0da3bb98afdb8596a9` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:04.156074 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef1a0f-7dba-4376-a80e-75b0190cfd9d None admin] RBAC: Authorizing `identity:get_user(user_id=ee9d005c534c4b0da3bb98afdb8596a9)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:04.157575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef1a0f-7dba-4376-a80e-75b0190cfd9d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:04.159344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 41/78] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:04 2022] GET /identity/v3/users/ee9d005c534c4b0da3bb98afdb8596a9 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:04.160119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef1a0f-7dba-4376-a80e-75b0190cfd9d None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:04.218171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc932a58-ec9b-44a4-b3cb-6235831bf29d None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:04.251575 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f40f94-d6c9-4430-b55c-a90054dffd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:04.252480 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f40f94-d6c9-4430-b55c-a90054dffd53 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:04.252776 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f40f94-d6c9-4430-b55c-a90054dffd53 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:04.252968 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:04.253081 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f40f94-d6c9-4430-b55c-a90054dffd53 None admin] PATH_INFO: `/v3/users/261a7891b0d244a3a8e71c73d6ba80b6` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:04.254227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f40f94-d6c9-4430-b55c-a90054dffd53 None admin] RBAC: Authorizing `identity:get_user(user_id=261a7891b0d244a3a8e71c73d6ba80b6)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:04.255721 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f40f94-d6c9-4430-b55c-a90054dffd53 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:04.257403 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 38/79] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:04 2022] GET /identity/v3/users/261a7891b0d244a3a8e71c73d6ba80b6 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:04.258175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f40f94-d6c9-4430-b55c-a90054dffd53 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:04.259789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef1a0f-7dba-4376-a80e-75b0190cfd9d None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:04.284825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:04.294846 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e4780-484f-4f59-a54a-838c7880af81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:04.295720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e4780-484f-4f59-a54a-838c7880af81 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:04.296007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e4780-484f-4f59-a54a-838c7880af81 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:04.296333 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e4780-484f-4f59-a54a-838c7880af81 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:04.307091 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e4780-484f-4f59-a54a-838c7880af81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:04.315035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-2f3e4780-484f-4f59-a54a-838c7880af81 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:04.315035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:04.315035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:04.315035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:04.315035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:04.315035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:04.315035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:04.315035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:04.315035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:04.315035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:04.315035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:04.315035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:04.315035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:04.315035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:04.315035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:04.315035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:04.315975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:04.315975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:04.315975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:04.315975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:04.315975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:04.315975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:04.315975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:04.315975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:04.315975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:04.315975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:04.315975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:04.315975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:04.315975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:04.315975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:04.315975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:04.315975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:04.316785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:04.316785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:04.316785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:04.316785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:04.316785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:04.316785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:04.316785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:04.316785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:04.316785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:04.316785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:04.316785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:04.316785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:04.316785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 42/80] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:04.317531 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e4780-484f-4f59-a54a-838c7880af81 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:04.358229 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f40f94-d6c9-4430-b55c-a90054dffd53 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:04.391990 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62873f3b-e056-4bbd-8c77-3c17a7bb0a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:04.392857 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62873f3b-e056-4bbd-8c77-3c17a7bb0a43 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:04.393147 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62873f3b-e056-4bbd-8c77-3c17a7bb0a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:04.393427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62873f3b-e056-4bbd-8c77-3c17a7bb0a43 None admin] PATH_INFO: `/v3/users/ee9d005c534c4b0da3bb98afdb8596a9` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:04.394760 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62873f3b-e056-4bbd-8c77-3c17a7bb0a43 None admin] RBAC: Authorizing `identity:get_user(user_id=ee9d005c534c4b0da3bb98afdb8596a9)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:04.396263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62873f3b-e056-4bbd-8c77-3c17a7bb0a43 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:04.397901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 39/81] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:04 2022] GET /identity/v3/users/ee9d005c534c4b0da3bb98afdb8596a9 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:04.402344 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62873f3b-e056-4bbd-8c77-3c17a7bb0a43 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:04.417878 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e4780-484f-4f59-a54a-838c7880af81 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:04.451311 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7b2bd-a774-425f-b2f4-a54e88ea2917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:04.452103 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7b2bd-a774-425f-b2f4-a54e88ea2917 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:04.452394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7b2bd-a774-425f-b2f4-a54e88ea2917 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:04.452678 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7b2bd-a774-425f-b2f4-a54e88ea2917 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:04.453854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d7b2bd-a774-425f-b2f4-a54e88ea2917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:04.455366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d7b2bd-a774-425f-b2f4-a54e88ea2917 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:04.457240 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 43/82] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:03:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:04.459743 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7b2bd-a774-425f-b2f4-a54e88ea2917 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:04.485747 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:04.492595 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:04.500281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62873f3b-e056-4bbd-8c77-3c17a7bb0a43 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:04.533927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4502333-7685-49c9-bd5a-5f475e907e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:04.534636 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4502333-7685-49c9-bd5a-5f475e907e93 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:04.534951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4502333-7685-49c9-bd5a-5f475e907e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:04.535239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4502333-7685-49c9-bd5a-5f475e907e93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:04.558992 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7b2bd-a774-425f-b2f4-a54e88ea2917 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:04.592260 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8a850-284f-469d-b89e-03c2bbcded64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:04.592914 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8a850-284f-469d-b89e-03c2bbcded64 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:04.593206 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8a850-284f-469d-b89e-03c2bbcded64 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:04.593488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8a850-284f-469d-b89e-03c2bbcded64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:04.601219 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4502333-7685-49c9-bd5a-5f475e907e93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:04.602704 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4502333-7685-49c9-bd5a-5f475e907e93 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:04.658901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8a850-284f-469d-b89e-03c2bbcded64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:04.660425 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8a850-284f-469d-b89e-03c2bbcded64 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:04.683195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:04.691101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:04.698057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 40/83] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:03:04.698867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4502333-7685-49c9-bd5a-5f475e907e93 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:04.757929 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 44/84] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:03:04.759142 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8a850-284f-469d-b89e-03c2bbcded64 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:04.799616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4502333-7685-49c9-bd5a-5f475e907e93 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:04.833064 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979182f-e44d-4d5e-a5a1-7be55d7e12a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:04.833744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979182f-e44d-4d5e-a5a1-7be55d7e12a4 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:04.834035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979182f-e44d-4d5e-a5a1-7be55d7e12a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:04.834321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979182f-e44d-4d5e-a5a1-7be55d7e12a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:04.835121 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979182f-e44d-4d5e-a5a1-7be55d7e12a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:04.835525 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979182f-e44d-4d5e-a5a1-7be55d7e12a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:04.836927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979182f-e44d-4d5e-a5a1-7be55d7e12a4 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:04.845232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 41/85] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:04.845798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979182f-e44d-4d5e-a5a1-7be55d7e12a4 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:04.859050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8a850-284f-469d-b89e-03c2bbcded64 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:04.881949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:04.893101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d91922-94c5-4693-b581-7081545f952a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:04.893752 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d91922-94c5-4693-b581-7081545f952a None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:04.894043 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d91922-94c5-4693-b581-7081545f952a None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:04.897277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d91922-94c5-4693-b581-7081545f952a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:04.902164 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:04.946142 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979182f-e44d-4d5e-a5a1-7be55d7e12a4 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:04.966387 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d91922-94c5-4693-b581-7081545f952a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:04.967918 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d91922-94c5-4693-b581-7081545f952a None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:04.979400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef264bc-ec1c-4a72-ac48-b6019b2c6c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:04.980083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef264bc-ec1c-4a72-ac48-b6019b2c6c4e None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:04.980374 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef264bc-ec1c-4a72-ac48-b6019b2c6c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:04.980658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef264bc-ec1c-4a72-ac48-b6019b2c6c4e None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:04.981925 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef264bc-ec1c-4a72-ac48-b6019b2c6c4e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:04.983418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef264bc-ec1c-4a72-ac48-b6019b2c6c4e None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:05.069440 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 45/86] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:03:05.070294 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d91922-94c5-4693-b581-7081545f952a None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:05.093575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:05.111859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 42/87] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jun 23 09:03:04 2022] POST /identity/v3/users => generated 289 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:03:05.111859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef264bc-ec1c-4a72-ac48-b6019b2c6c4e None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:05.183908 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d91922-94c5-4693-b581-7081545f952a None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:05.223247 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1c112-7ccd-4e86-9321-5aca74905075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:05.224146 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1c112-7ccd-4e86-9321-5aca74905075 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:05.224441 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1c112-7ccd-4e86-9321-5aca74905075 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:05.224735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1c112-7ccd-4e86-9321-5aca74905075 None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:05.226038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc1c112-7ccd-4e86-9321-5aca74905075 None admin] RBAC: Authorizing `identity:get_project(project_id=79d2a60c71124efc90e0d0b445517d55)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:05.227578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc1c112-7ccd-4e86-9321-5aca74905075 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:05.234711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 46/88] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:05 2022] GET /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:05.234711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1c112-7ccd-4e86-9321-5aca74905075 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:05.244522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef264bc-ec1c-4a72-ac48-b6019b2c6c4e None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:05.259615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:05.288519 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72cf66a-c808-43cd-895a-cd690d9d4414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:05.289489 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cf66a-c808-43cd-895a-cd690d9d4414 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:05.289811 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cf66a-c808-43cd-895a-cd690d9d4414 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:05.290099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cf66a-c808-43cd-895a-cd690d9d4414 None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:05.292470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72cf66a-c808-43cd-895a-cd690d9d4414 None admin] RBAC: Authorizing `identity:get_project(project_id=79d2a60c71124efc90e0d0b445517d55)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:05.294946 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72cf66a-c808-43cd-895a-cd690d9d4414 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:05.296813 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 43/89] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:05 2022] GET /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:05.306971 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72cf66a-c808-43cd-895a-cd690d9d4414 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:05.339695 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:05.351757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1c112-7ccd-4e86-9321-5aca74905075 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:05.403825 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e6109-4dc2-4b7d-83d5-e146c65bb9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:05.404845 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e6109-4dc2-4b7d-83d5-e146c65bb9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:05.405124 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e6109-4dc2-4b7d-83d5-e146c65bb9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:05.405410 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e6109-4dc2-4b7d-83d5-e146c65bb9b7 None admin] PATH_INFO: `/v3/users/ee9d005c534c4b0da3bb98afdb8596a9` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:05.407384 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e6109-4dc2-4b7d-83d5-e146c65bb9b7 None admin] RBAC: Authorizing `identity:get_user(user_id=ee9d005c534c4b0da3bb98afdb8596a9)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:05.408990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e6109-4dc2-4b7d-83d5-e146c65bb9b7 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:05.410673 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 47/90] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:05 2022] GET /identity/v3/users/ee9d005c534c4b0da3bb98afdb8596a9 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:05.411668 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e6109-4dc2-4b7d-83d5-e146c65bb9b7 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:05.443047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72cf66a-c808-43cd-895a-cd690d9d4414 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:05.491384 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35c5ed-bed1-4397-8d0a-44389c1a3d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:05.492527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35c5ed-bed1-4397-8d0a-44389c1a3d8e None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:05.492823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35c5ed-bed1-4397-8d0a-44389c1a3d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:05.493104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35c5ed-bed1-4397-8d0a-44389c1a3d8e None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:05.495011 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35c5ed-bed1-4397-8d0a-44389c1a3d8e None admin] RBAC: Authorizing `identity:get_project(project_id=79d2a60c71124efc90e0d0b445517d55)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:05.496761 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35c5ed-bed1-4397-8d0a-44389c1a3d8e None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:05.499493 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 44/91] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:05 2022] GET /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:05.505185 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35c5ed-bed1-4397-8d0a-44389c1a3d8e None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:05.554211 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e6109-4dc2-4b7d-83d5-e146c65bb9b7 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:05.564235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:05.605017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a07f7-2b61-422d-a27b-4987d822b50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:05.605611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a07f7-2b61-422d-a27b-4987d822b50b None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:05.605903 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a07f7-2b61-422d-a27b-4987d822b50b None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:05.606685 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a07f7-2b61-422d-a27b-4987d822b50b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:05.607590 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a07f7-2b61-422d-a27b-4987d822b50b None admin] RBAC: Adding query filter params (role.id=53a3d65343394e7caafcc33f12cf138b, scope.project.id=79d2a60c71124efc90e0d0b445517d55, user.id=ee9d005c534c4b0da3bb98afdb8596a9) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:05.607901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a07f7-2b61-422d-a27b-4987d822b50b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:05.609357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a07f7-2b61-422d-a27b-4987d822b50b None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:05.617699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 48/92] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:03:05 2022] GET /identity/v3/role_assignments?user.id=ee9d005c534c4b0da3bb98afdb8596a9&scope.project.id=79d2a60c71124efc90e0d0b445517d55&role.id=53a3d65343394e7caafcc33f12cf138b => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:05.618726 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a07f7-2b61-422d-a27b-4987d822b50b None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:05.624722 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35c5ed-bed1-4397-8d0a-44389c1a3d8e None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:05.652539 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:05.664784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465de50-f775-4b96-a73c-524aa0ddee0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:05.666103 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465de50-f775-4b96-a73c-524aa0ddee0c None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:05.666614 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465de50-f775-4b96-a73c-524aa0ddee0c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:05.674947 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465de50-f775-4b96-a73c-524aa0ddee0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:05.674947 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465de50-f775-4b96-a73c-524aa0ddee0c None admin] RBAC: Adding query filter params (role.id=9cd080b50c6f4ee8a54bf9ef1655bd83, scope.project.id=79d2a60c71124efc90e0d0b445517d55, user.id=261a7891b0d244a3a8e71c73d6ba80b6) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:05.674947 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465de50-f775-4b96-a73c-524aa0ddee0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:05.674947 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465de50-f775-4b96-a73c-524aa0ddee0c None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:05.680232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 45/93] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:03:05 2022] GET /identity/v3/role_assignments?user.id=261a7891b0d244a3a8e71c73d6ba80b6&scope.project.id=79d2a60c71124efc90e0d0b445517d55&role.id=9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:05.681433 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465de50-f775-4b96-a73c-524aa0ddee0c None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:05.747841 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a07f7-2b61-422d-a27b-4987d822b50b None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:05.787249 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ef9c4-a7b1-4cf6-b44a-e316159b7aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:05.787944 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ef9c4-a7b1-4cf6-b44a-e316159b7aee None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:05.788236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ef9c4-a7b1-4cf6-b44a-e316159b7aee None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:05.788514 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ef9c4-a7b1-4cf6-b44a-e316159b7aee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:05.803295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465de50-f775-4b96-a73c-524aa0ddee0c None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:05.819026 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:05.844737 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b1b1f-af71-41d0-b599-e5fa095dcd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:05.846964 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b1b1f-af71-41d0-b599-e5fa095dcd34 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:05.847578 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b1b1f-af71-41d0-b599-e5fa095dcd34 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:05.848110 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b1b1f-af71-41d0-b599-e5fa095dcd34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:05.849268 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b1b1f-af71-41d0-b599-e5fa095dcd34 None admin] RBAC: Adding query filter params (role.id=478b327fd22845cc98a2fafe821b1e85, scope.project.id=79d2a60c71124efc90e0d0b445517d55, user.id=ee9d005c534c4b0da3bb98afdb8596a9) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:05.849829 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b1b1f-af71-41d0-b599-e5fa095dcd34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:05.851768 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b1b1f-af71-41d0-b599-e5fa095dcd34 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:05.859211 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 46/94] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:03:05 2022] GET /identity/v3/role_assignments?user.id=ee9d005c534c4b0da3bb98afdb8596a9&scope.project.id=79d2a60c71124efc90e0d0b445517d55&role.id=478b327fd22845cc98a2fafe821b1e85 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:05.871101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ef9c4-a7b1-4cf6-b44a-e316159b7aee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:05.873169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ef9c4-a7b1-4cf6-b44a-e316159b7aee None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:05.915930 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:06.013916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 49/95] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:03:06.019919 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b1b1f-af71-41d0-b599-e5fa095dcd34 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:06.142287 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b1b1f-af71-41d0-b599-e5fa095dcd34 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:06.180210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871c223-1146-48b5-bdc5-6876cccab975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:06.181098 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871c223-1146-48b5-bdc5-6876cccab975 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:06.181391 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871c223-1146-48b5-bdc5-6876cccab975 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:06.181680 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871c223-1146-48b5-bdc5-6876cccab975 None admin] PATH_INFO: `/v3/projects/5789528db5bf4c14a1dc84d2cefd203f` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:06.182898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871c223-1146-48b5-bdc5-6876cccab975 None admin] RBAC: Authorizing `identity:get_project(project_id=5789528db5bf4c14a1dc84d2cefd203f)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:06.184475 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871c223-1146-48b5-bdc5-6876cccab975 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:06.186543 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 47/96] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:06 2022] GET /identity/v3/projects/5789528db5bf4c14a1dc84d2cefd203f => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:06.196059 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ef9c4-a7b1-4cf6-b44a-e316159b7aee None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:06.306399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ef9c4-a7b1-4cf6-b44a-e316159b7aee None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:06.341221 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:06.359653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f055463-0013-4266-befa-940c7d50c513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:06.360321 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f055463-0013-4266-befa-940c7d50c513 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:06.360615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f055463-0013-4266-befa-940c7d50c513 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:06.360905 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f055463-0013-4266-befa-940c7d50c513 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:06.361754 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f055463-0013-4266-befa-940c7d50c513 None admin] RBAC: Adding query filter params (role.id=53a3d65343394e7caafcc33f12cf138b, scope.project.id=5789528db5bf4c14a1dc84d2cefd203f, user.id=ee9d005c534c4b0da3bb98afdb8596a9) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:06.362072 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f055463-0013-4266-befa-940c7d50c513 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:06.363593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f055463-0013-4266-befa-940c7d50c513 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:06.369721 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 50/97] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:03:06 2022] GET /identity/v3/role_assignments?user.id=ee9d005c534c4b0da3bb98afdb8596a9&scope.project.id=5789528db5bf4c14a1dc84d2cefd203f&role.id=53a3d65343394e7caafcc33f12cf138b => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:08.281911 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2520db-ee25-40db-ac0f-101fc6472400 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:08.282414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2520db-ee25-40db-ac0f-101fc6472400 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:08.282490 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2520db-ee25-40db-ac0f-101fc6472400 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:08.284269 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 48/98] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:08.289638 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f63b1-327f-4823-a1f8-d65e5637f551 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:08.290023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f63b1-327f-4823-a1f8-d65e5637f551 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:08.290215 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f63b1-327f-4823-a1f8-d65e5637f551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:08.307722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-1f4f63b1-327f-4823-a1f8-d65e5637f551 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:08.308919 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-1f4f63b1-327f-4823-a1f8-d65e5637f551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:08.395206 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 51/99] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:08.428514 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2520db-ee25-40db-ac0f-101fc6472400 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:08.490253 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:08.564086 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2520db-ee25-40db-ac0f-101fc6472400 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:08.612357 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944dcb8-3848-43c8-9839-05ce22c25147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:08.613179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944dcb8-3848-43c8-9839-05ce22c25147 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:08.613476 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944dcb8-3848-43c8-9839-05ce22c25147 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:08.613763 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944dcb8-3848-43c8-9839-05ce22c25147 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:08.624618 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944dcb8-3848-43c8-9839-05ce22c25147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:08.636599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-5944dcb8-3848-43c8-9839-05ce22c25147 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 09:03:08.636599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:08.636599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:08.636599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:08.636599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:08.636599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:08.636599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:08.636599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:08.636599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:08.636599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:08.636599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:08.636599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:08.636599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:08.636599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:08.636599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:08.636599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:08.637527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:08.637527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:08.637527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:08.637527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:08.637527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:08.637527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:08.637527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:08.637527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:08.637527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:08.637527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:08.637527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:08.637527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:08.637527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:08.637527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:08.637527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:08.637527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:08.638326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:08.638326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:08.638326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:08.638326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:08.638326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:08.638326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:08.638326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:08.638326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:08.638326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:08.638326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:08.638326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 09:03:08.638326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:08.638326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 49/100] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:08.643830 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f63b1-327f-4823-a1f8-d65e5637f551 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:08.772416 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f63b1-327f-4823-a1f8-d65e5637f551 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:08.799351 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:08.816521 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d6d04-739e-4376-88bd-8ab8a26d9cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:08.817268 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d6d04-739e-4376-88bd-8ab8a26d9cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:08.817564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d6d04-739e-4376-88bd-8ab8a26d9cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:08.817850 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d6d04-739e-4376-88bd-8ab8a26d9cb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:08.818619 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d6d04-739e-4376-88bd-8ab8a26d9cb0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:08.823222 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d6d04-739e-4376-88bd-8ab8a26d9cb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:08.824540 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d6d04-739e-4376-88bd-8ab8a26d9cb0 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:08.824790 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d942f76-84c3-4474-ba2c-41ff362c2f7b None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:08.825158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d942f76-84c3-4474-ba2c-41ff362c2f7b None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:08.825472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d942f76-84c3-4474-ba2c-41ff362c2f7b None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:08.827691 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 50/101] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:08.834488 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107832a-5354-4899-902f-baf6b5777d67 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:08.834795 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107832a-5354-4899-902f-baf6b5777d67 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:08.835115 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107832a-5354-4899-902f-baf6b5777d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:08.842024 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 52/102] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:08.843497 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d734928-ec7f-41b0-8b58-34b90bf90ce9 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:08.843918 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d734928-ec7f-41b0-8b58-34b90bf90ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:08.844213 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d734928-ec7f-41b0-8b58-34b90bf90ce9 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:08.845882 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 53/103] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:08.851116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b57c5-a299-459b-8792-a653b7b01eaa None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:08.851530 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b57c5-a299-459b-8792-a653b7b01eaa None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:08.851921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b57c5-a299-459b-8792-a653b7b01eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:08.863047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-4107832a-5354-4899-902f-baf6b5777d67 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:08.864177 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-4107832a-5354-4899-902f-baf6b5777d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:08.890258 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-c93b57c5-a299-459b-8792-a653b7b01eaa None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:08.891558 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-c93b57c5-a299-459b-8792-a653b7b01eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:08.980681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 51/104] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:08.982277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107832a-5354-4899-902f-baf6b5777d67 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:09.003628 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 54/105] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:03:09.005393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfbbda-a45a-4457-af7b-898bdd8f6a95 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:09.005698 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfbbda-a45a-4457-af7b-898bdd8f6a95 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:09.005987 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfbbda-a45a-4457-af7b-898bdd8f6a95 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:09.007872 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 55/106] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:09.013982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33c6d2-533c-4ef2-a358-2862121fd064 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:09.014349 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33c6d2-533c-4ef2-a358-2862121fd064 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:09.014638 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33c6d2-533c-4ef2-a358-2862121fd064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:09.025658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:09.032865 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-7b33c6d2-533c-4ef2-a358-2862121fd064 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:09.034427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-7b33c6d2-533c-4ef2-a358-2862121fd064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:09.107753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107832a-5354-4899-902f-baf6b5777d67 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:09.114506 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 56/107] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:03:09.123913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b33c6d2-533c-4ef2-a358-2862121fd064 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:09.155083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca66530-8cb1-4e6c-85b0-076751f9b725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:09.156813 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca66530-8cb1-4e6c-85b0-076751f9b725 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:09.157105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca66530-8cb1-4e6c-85b0-076751f9b725 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:09.157387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca66530-8cb1-4e6c-85b0-076751f9b725 None admin] PATH_INFO: `/v3/users/ca520b6a862c447987cd8f3b34dc58e7` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:09.188088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca66530-8cb1-4e6c-85b0-076751f9b725 None admin] RBAC: Authorizing `identity:get_user(user_id=ca520b6a862c447987cd8f3b34dc58e7)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:09.190101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:09.191791 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca66530-8cb1-4e6c-85b0-076751f9b725 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:09.195415 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 52/108] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:08 2022] GET /identity/v3/users/ca520b6a862c447987cd8f3b34dc58e7 => generated 289 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:09.196613 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca66530-8cb1-4e6c-85b0-076751f9b725 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:09.259011 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b33c6d2-533c-4ef2-a358-2862121fd064 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:09.294728 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d8290-e9a3-4f51-b89b-937623545530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:09.295499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d8290-e9a3-4f51-b89b-937623545530 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:09.295784 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d8290-e9a3-4f51-b89b-937623545530 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:09.296069 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d8290-e9a3-4f51-b89b-937623545530 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:09.297262 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d8290-e9a3-4f51-b89b-937623545530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:09.298752 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d8290-e9a3-4f51-b89b-937623545530 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:09.300599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 57/109] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:03:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:09.301281 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d8290-e9a3-4f51-b89b-937623545530 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:09.303563 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:09.312342 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca66530-8cb1-4e6c-85b0-076751f9b725 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:09.351815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5378a1-9a95-48ed-a216-c42c90bbbe47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:09.352643 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5378a1-9a95-48ed-a216-c42c90bbbe47 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:09.352942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5378a1-9a95-48ed-a216-c42c90bbbe47 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:09.353223 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5378a1-9a95-48ed-a216-c42c90bbbe47 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:09.364305 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5378a1-9a95-48ed-a216-c42c90bbbe47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:09.372874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-da5378a1-9a95-48ed-a216-c42c90bbbe47 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:09.372874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:09.372874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:09.372874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:09.372874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:09.372874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:09.372874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:09.372874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:09.372874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:09.372874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:09.372874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:09.372874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:09.372874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:09.372874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:09.372874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:09.372874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:09.373805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:09.373805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:09.373805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:09.373805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:09.373805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:09.373805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:09.373805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:09.373805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:09.373805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:09.373805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:09.373805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:09.373805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:09.373805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:09.373805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:09.373805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:09.373805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:09.374703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:09.374703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:09.374703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:09.374703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:09.374703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:09.374703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:09.374703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:09.374703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:09.374703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:09.374703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:09.374703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:09.374703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:09.374703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 53/110] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:03:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:09.376041 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5378a1-9a95-48ed-a216-c42c90bbbe47 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:09.419688 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d8290-e9a3-4f51-b89b-937623545530 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:09.428062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:09.455133 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2dd21-4d06-448c-aca6-5e391ec872e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:09.456302 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2dd21-4d06-448c-aca6-5e391ec872e2 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:09.456666 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2dd21-4d06-448c-aca6-5e391ec872e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:09.456959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2dd21-4d06-448c-aca6-5e391ec872e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:09.468230 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2dd21-4d06-448c-aca6-5e391ec872e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:09.476157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-7ab2dd21-4d06-448c-aca6-5e391ec872e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:03:09.476157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:09.476157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:09.476157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:09.476157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:09.476157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:09.476157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:09.476157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:09.476157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:09.476157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:09.476157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:09.476157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:09.476157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:09.476157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:09.476157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:09.476157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:09.477153 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:09.477153 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:09.477153 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:09.477153 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:09.477153 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:09.477153 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:09.477153 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:09.477153 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:09.477153 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:09.477153 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:09.477153 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:09.477153 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:09.477153 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:09.477153 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:09.477153 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:09.477153 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:09.477956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:09.477956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:09.477956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:09.477956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:09.477956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:09.477956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:09.477956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:09.477956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:09.477956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:09.477956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:09.477956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:03:09.477956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:09.480878 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 58/111] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:03:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:09.482203 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293917f-c8c5-4b47-b6c6-b6e39d334f52 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:09.483138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293917f-c8c5-4b47-b6c6-b6e39d334f52 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:09.483329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293917f-c8c5-4b47-b6c6-b6e39d334f52 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:09.484586 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 59/112] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:09.487170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293917f-c8c5-4b47-b6c6-b6e39d334f52 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:09.495262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5378a1-9a95-48ed-a216-c42c90bbbe47 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:09.532025 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b485d85-99fa-480d-aa76-e5d7c396f0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:09.532689 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b485d85-99fa-480d-aa76-e5d7c396f0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:09.532990 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b485d85-99fa-480d-aa76-e5d7c396f0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:09.533282 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b485d85-99fa-480d-aa76-e5d7c396f0d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:09.554861 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:09.589720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293917f-c8c5-4b47-b6c6-b6e39d334f52 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:09.600765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b485d85-99fa-480d-aa76-e5d7c396f0d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:09.602326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b485d85-99fa-480d-aa76-e5d7c396f0d2 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:09.623839 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936cd17-b8d7-4232-9614-3ee097cc3486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:09.624510 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936cd17-b8d7-4232-9614-3ee097cc3486 None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:09.624806 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936cd17-b8d7-4232-9614-3ee097cc3486 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:09.625097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936cd17-b8d7-4232-9614-3ee097cc3486 None admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:09.625816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1936cd17-b8d7-4232-9614-3ee097cc3486 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:09.627317 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1936cd17-b8d7-4232-9614-3ee097cc3486 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:09.697593 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 54/113] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:03:09.698431 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b485d85-99fa-480d-aa76-e5d7c396f0d2 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:09.725052 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 60/114] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jun 23 09:03:09 2022] POST /identity/v3/projects => generated 302 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:03:09.725975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936cd17-b8d7-4232-9614-3ee097cc3486 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:09.760165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:09.767163 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:09.812638 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b485d85-99fa-480d-aa76-e5d7c396f0d2 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:09.855221 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f0d32-095d-441d-8924-ded98a6c19a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:09.856318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f0d32-095d-441d-8924-ded98a6c19a4 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:09.856764 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936cd17-b8d7-4232-9614-3ee097cc3486 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:09.857025 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f0d32-095d-441d-8924-ded98a6c19a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:09.857533 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f0d32-095d-441d-8924-ded98a6c19a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:09.858704 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f0d32-095d-441d-8924-ded98a6c19a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:09.859921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f0d32-095d-441d-8924-ded98a6c19a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:09.861437 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f0d32-095d-441d-8924-ded98a6c19a4 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:09.874943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 55/115] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:03:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:09.874943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51dd5c1-ce5b-4742-89a3-95fc9b5105b5 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:09.874943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51dd5c1-ce5b-4742-89a3-95fc9b5105b5 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:09.874943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51dd5c1-ce5b-4742-89a3-95fc9b5105b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:09.891325 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-d51dd5c1-ce5b-4742-89a3-95fc9b5105b5 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:09.892537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-d51dd5c1-ce5b-4742-89a3-95fc9b5105b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:09.901453 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905161f7-3c1c-4b6b-9c1e-8659cf35d21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:09.902141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905161f7-3c1c-4b6b-9c1e-8659cf35d21e None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:09.902430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905161f7-3c1c-4b6b-9c1e-8659cf35d21e None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:09.903451 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905161f7-3c1c-4b6b-9c1e-8659cf35d21e None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:09.903662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905161f7-3c1c-4b6b-9c1e-8659cf35d21e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:09.904114 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905161f7-3c1c-4b6b-9c1e-8659cf35d21e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:09.905284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905161f7-3c1c-4b6b-9c1e-8659cf35d21e None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:09.916913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 61/116] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:09.918242 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b90230-6757-4766-8108-2baad5d820fc None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:09.918544 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b90230-6757-4766-8108-2baad5d820fc None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:09.921120 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b90230-6757-4766-8108-2baad5d820fc None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:09.921322 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 62/117] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:09.922031 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b90230-6757-4766-8108-2baad5d820fc None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:09.969509 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 56/118] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:09.970504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51dd5c1-ce5b-4742-89a3-95fc9b5105b5 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:10.019776 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:10.025046 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:10.041990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b90230-6757-4766-8108-2baad5d820fc None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:10.078577 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51dd5c1-ce5b-4742-89a3-95fc9b5105b5 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:10.078745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273279df-463e-406f-b749-0f8d79b5aef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:10.079670 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273279df-463e-406f-b749-0f8d79b5aef7 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:10.079970 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273279df-463e-406f-b749-0f8d79b5aef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:10.080256 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273279df-463e-406f-b749-0f8d79b5aef7 None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:10.081508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273279df-463e-406f-b749-0f8d79b5aef7 None admin] RBAC: Authorizing `identity:get_project(project_id=79d2a60c71124efc90e0d0b445517d55)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:10.083332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273279df-463e-406f-b749-0f8d79b5aef7 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:10.085251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 63/119] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:09 2022] GET /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:10.085857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273279df-463e-406f-b749-0f8d79b5aef7 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:10.120358 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b493b6-1dd2-4d73-a018-204dd401a3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:10.121796 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b493b6-1dd2-4d73-a018-204dd401a3a0 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:10.122238 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b493b6-1dd2-4d73-a018-204dd401a3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:10.123292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b493b6-1dd2-4d73-a018-204dd401a3a0 None admin] PATH_INFO: `/v3/users/ee9d005c534c4b0da3bb98afdb8596a9` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:10.124752 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b493b6-1dd2-4d73-a018-204dd401a3a0 None admin] RBAC: Authorizing `identity:get_user(user_id=ee9d005c534c4b0da3bb98afdb8596a9)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:10.126512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b493b6-1dd2-4d73-a018-204dd401a3a0 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:10.132942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 57/120] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:09 2022] GET /identity/v3/users/ee9d005c534c4b0da3bb98afdb8596a9 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:10.132942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe2839-10cb-4d6a-acd2-f012ac7f6626 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:10.132942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe2839-10cb-4d6a-acd2-f012ac7f6626 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:10.132942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe2839-10cb-4d6a-acd2-f012ac7f6626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:10.152689 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-8bfe2839-10cb-4d6a-acd2-f012ac7f6626 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:10.154568 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-8bfe2839-10cb-4d6a-acd2-f012ac7f6626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:10.195076 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273279df-463e-406f-b749-0f8d79b5aef7 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:10.229400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 58/121] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:10.230256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe2839-10cb-4d6a-acd2-f012ac7f6626 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:10.245943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f385b54-a612-4357-878a-fac3115a6515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:10.247646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f385b54-a612-4357-878a-fac3115a6515 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:10.247941 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f385b54-a612-4357-878a-fac3115a6515 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:10.248402 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f385b54-a612-4357-878a-fac3115a6515 None admin] PATH_INFO: `/v3/users/261a7891b0d244a3a8e71c73d6ba80b6` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:10.249919 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f385b54-a612-4357-878a-fac3115a6515 None admin] RBAC: Authorizing `identity:get_user(user_id=261a7891b0d244a3a8e71c73d6ba80b6)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:10.251531 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f385b54-a612-4357-878a-fac3115a6515 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:10.254198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 64/122] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:10 2022] GET /identity/v3/users/261a7891b0d244a3a8e71c73d6ba80b6 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:10.254988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f385b54-a612-4357-878a-fac3115a6515 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:10.274292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:10.276926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:10.330667 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe2839-10cb-4d6a-acd2-f012ac7f6626 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:10.360393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f385b54-a612-4357-878a-fac3115a6515 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:10.366452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a2b9f-ca9d-4484-84ae-404838511e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:10.367285 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a2b9f-ca9d-4484-84ae-404838511e81 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:10.367632 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a2b9f-ca9d-4484-84ae-404838511e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:10.367913 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a2b9f-ca9d-4484-84ae-404838511e81 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:10.380245 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a2b9f-ca9d-4484-84ae-404838511e81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:10.388222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-934a2b9f-ca9d-4484-84ae-404838511e81 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:10.388222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:10.388222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:10.388222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:10.388222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:10.388222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:10.388222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:10.388222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:10.388222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:10.388222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:10.388222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:10.388222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:10.388222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:10.388222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:10.388222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:10.388222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:10.389232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:10.389232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:10.389232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:10.389232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:10.389232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:10.389232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:10.389232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:10.389232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:10.389232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:10.389232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:10.389232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:10.389232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:10.389232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:10.389232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:10.389232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:10.389232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:10.390042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:10.390042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:10.390042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:10.390042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:10.390042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:10.390042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:10.390042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:10.390042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:10.390042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:10.390042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:10.390042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:10.390042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:10.390042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 59/123] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:10.390710 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a2b9f-ca9d-4484-84ae-404838511e81 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:10.406027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1e53e-bf1c-4956-a8c7-ceb0a7d13854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:10.407016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1e53e-bf1c-4956-a8c7-ceb0a7d13854 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:10.407317 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1e53e-bf1c-4956-a8c7-ceb0a7d13854 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:10.407606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1e53e-bf1c-4956-a8c7-ceb0a7d13854 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:10.408625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1e53e-bf1c-4956-a8c7-ceb0a7d13854 None admin] RBAC: Adding query filter params (role.id=a6f4585744cf42e6b971891968972799, scope.project.id=79d2a60c71124efc90e0d0b445517d55, user.id=ca520b6a862c447987cd8f3b34dc58e7) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:10.409321 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1e53e-bf1c-4956-a8c7-ceb0a7d13854 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:10.411398 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1e53e-bf1c-4956-a8c7-ceb0a7d13854 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:10.417507 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 65/124] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:03:10 2022] GET /identity/v3/role_assignments?user.id=ca520b6a862c447987cd8f3b34dc58e7&scope.project.id=79d2a60c71124efc90e0d0b445517d55&role.id=a6f4585744cf42e6b971891968972799 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:10.418302 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1e53e-bf1c-4956-a8c7-ceb0a7d13854 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:10.525049 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:10.530681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a2b9f-ca9d-4484-84ae-404838511e81 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:10.532544 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:10.554581 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1e53e-bf1c-4956-a8c7-ceb0a7d13854 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:10.570195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a187806-4d7e-4214-a816-b3d7d5beb8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:10.571854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a187806-4d7e-4214-a816-b3d7d5beb8fb None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:10.572236 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a187806-4d7e-4214-a816-b3d7d5beb8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:10.572564 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a187806-4d7e-4214-a816-b3d7d5beb8fb None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:10.573837 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a187806-4d7e-4214-a816-b3d7d5beb8fb None admin] RBAC: Authorizing `identity:get_project(project_id=79d2a60c71124efc90e0d0b445517d55)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:10.576011 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a187806-4d7e-4214-a816-b3d7d5beb8fb None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:10.578035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 60/125] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:10 2022] GET /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:10.578557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a187806-4d7e-4214-a816-b3d7d5beb8fb None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:10.607873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0a766-c95d-4b96-b597-8baafeb4f8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:10.609086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0a766-c95d-4b96-b597-8baafeb4f8bc None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:10.609508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0a766-c95d-4b96-b597-8baafeb4f8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:10.609796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0a766-c95d-4b96-b597-8baafeb4f8bc None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:10.611339 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0a766-c95d-4b96-b597-8baafeb4f8bc None admin] RBAC: Authorizing `identity:get_project(project_id=79d2a60c71124efc90e0d0b445517d55)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:10.613495 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0a766-c95d-4b96-b597-8baafeb4f8bc None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:10.615768 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 66/126] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:10 2022] GET /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:10.619678 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0a766-c95d-4b96-b597-8baafeb4f8bc None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:10.703525 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a187806-4d7e-4214-a816-b3d7d5beb8fb None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:10.723156 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0a766-c95d-4b96-b597-8baafeb4f8bc None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:10.741669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfda5de-08e5-44ac-a9e6-463cbf18b898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:10.742475 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfda5de-08e5-44ac-a9e6-463cbf18b898 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:10.742816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfda5de-08e5-44ac-a9e6-463cbf18b898 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:10.743131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfda5de-08e5-44ac-a9e6-463cbf18b898 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:10.754561 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfda5de-08e5-44ac-a9e6-463cbf18b898 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:10.759455 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6ef4f-8198-4810-a2af-70045f3afb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:10.760354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6ef4f-8198-4810-a2af-70045f3afb68 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:10.760842 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6ef4f-8198-4810-a2af-70045f3afb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:10.761336 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6ef4f-8198-4810-a2af-70045f3afb68 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:10.762295 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6ef4f-8198-4810-a2af-70045f3afb68 None admin] RBAC: Adding query filter params (name=member) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:10.763003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-7dfda5de-08e5-44ac-a9e6-463cbf18b898 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:10.763003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:10.763003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:10.763003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:10.763003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:10.763003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:10.763003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:10.763003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:10.763003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:10.763003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:10.763003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:10.763003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:10.763003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:10.763003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:10.763003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:10.763003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:10.763878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:10.763878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:10.763878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:10.763878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:10.763878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:10.763878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:10.763878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:10.763878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:10.763878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:10.763878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:10.763878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:10.763878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:10.763878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:10.763878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:10.763878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:10.763878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:10.764702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:10.764702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:10.764702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:10.764702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:10.764702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:10.764702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:10.764702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:10.764702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:10.764702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:10.764702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:10.764702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:10.764702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:10.764702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 61/127] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:10.765365 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfda5de-08e5-44ac-a9e6-463cbf18b898 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:10.766501 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6ef4f-8198-4810-a2af-70045f3afb68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:10.768154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6ef4f-8198-4810-a2af-70045f3afb68 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:10.777251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 67/128] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:10.778664 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6ef4f-8198-4810-a2af-70045f3afb68 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:10.791336 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:10.806237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:10.883077 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6ef4f-8198-4810-a2af-70045f3afb68 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:10.888427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfda5de-08e5-44ac-a9e6-463cbf18b898 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:10.922875 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d6c89-c0c2-4dde-acdb-e21b29dcaf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:10.923977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d6c89-c0c2-4dde-acdb-e21b29dcaf20 None admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:10.924343 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d6c89-c0c2-4dde-acdb-e21b29dcaf20 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:10.924732 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d6c89-c0c2-4dde-acdb-e21b29dcaf20 None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55/users/261a7891b0d244a3a8e71c73d6ba80b6/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:10.925504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ece03-160b-473d-a694-7279dda616bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:10.926555 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ece03-160b-473d-a694-7279dda616bd None admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:10.926976 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d6c89-c0c2-4dde-acdb-e21b29dcaf20 None admin] RBAC: Authorizing `identity:create_grant(project_id=79d2a60c71124efc90e0d0b445517d55, user_id=261a7891b0d244a3a8e71c73d6ba80b6, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:10.927152 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ece03-160b-473d-a694-7279dda616bd None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:10.927584 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ece03-160b-473d-a694-7279dda616bd None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55/users/ee9d005c534c4b0da3bb98afdb8596a9/roles/478b327fd22845cc98a2fafe821b1e85` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:10.928703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d6c89-c0c2-4dde-acdb-e21b29dcaf20 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:10.930143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ece03-160b-473d-a694-7279dda616bd None admin] RBAC: Authorizing `identity:create_grant(project_id=79d2a60c71124efc90e0d0b445517d55, user_id=ee9d005c534c4b0da3bb98afdb8596a9, role_id=478b327fd22845cc98a2fafe821b1e85)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:10.931757 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ece03-160b-473d-a694-7279dda616bd None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:11.028665 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 68/129] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:03:10 2022] PUT /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55/users/261a7891b0d244a3a8e71c73d6ba80b6/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:03:11.028665 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d6c89-c0c2-4dde-acdb-e21b29dcaf20 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:11.065987 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 62/130] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:03:10 2022] PUT /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55/users/ee9d005c534c4b0da3bb98afdb8596a9/roles/478b327fd22845cc98a2fafe821b1e85 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:03:11.067363 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ece03-160b-473d-a694-7279dda616bd None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:11.165143 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:11.266757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d6c89-c0c2-4dde-acdb-e21b29dcaf20 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:11.322449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86584bc-5933-420d-879d-dc7505f17680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:11.323168 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86584bc-5933-420d-879d-dc7505f17680 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:11.323459 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86584bc-5933-420d-879d-dc7505f17680 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:11.323739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86584bc-5933-420d-879d-dc7505f17680 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:11.324511 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86584bc-5933-420d-879d-dc7505f17680 None admin] RBAC: Adding query filter params (name=member) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:11.324820 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86584bc-5933-420d-879d-dc7505f17680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:11.326771 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86584bc-5933-420d-879d-dc7505f17680 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:11.340734 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 69/131] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:11.347271 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86584bc-5933-420d-879d-dc7505f17680 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:11.353808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:11.377182 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ece03-160b-473d-a694-7279dda616bd None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:11.473498 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86584bc-5933-420d-879d-dc7505f17680 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:11.503648 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b3f3f-d9f1-47b9-b475-953adccfcb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:11.504566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b3f3f-d9f1-47b9-b475-953adccfcb62 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:11.504866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b3f3f-d9f1-47b9-b475-953adccfcb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:11.505157 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b3f3f-d9f1-47b9-b475-953adccfcb62 None admin] PATH_INFO: `/v3/users/ee9d005c534c4b0da3bb98afdb8596a9` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:11.508647 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b3f3f-d9f1-47b9-b475-953adccfcb62 None admin] RBAC: Authorizing `identity:get_user(user_id=ee9d005c534c4b0da3bb98afdb8596a9)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:11.510328 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b3f3f-d9f1-47b9-b475-953adccfcb62 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:11.510735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec1ad7-f503-4cb0-9011-7c90efad5a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:11.511733 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec1ad7-f503-4cb0-9011-7c90efad5a33 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:11.512029 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec1ad7-f503-4cb0-9011-7c90efad5a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:11.512135 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 63/132] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:11 2022] GET /identity/v3/users/ee9d005c534c4b0da3bb98afdb8596a9 => generated 275 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:11.512347 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec1ad7-f503-4cb0-9011-7c90efad5a33 None admin] PATH_INFO: `/v3/users/ee9d005c534c4b0da3bb98afdb8596a9` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:11.513587 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec1ad7-f503-4cb0-9011-7c90efad5a33 None admin] RBAC: Authorizing `identity:get_user(user_id=ee9d005c534c4b0da3bb98afdb8596a9)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:11.514063 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:11.517628 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b3f3f-d9f1-47b9-b475-953adccfcb62 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:11.518188 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec1ad7-f503-4cb0-9011-7c90efad5a33 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:11.520086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 70/133] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:11 2022] GET /identity/v3/users/ee9d005c534c4b0da3bb98afdb8596a9 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:11.525667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec1ad7-f503-4cb0-9011-7c90efad5a33 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:11.652616 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec1ad7-f503-4cb0-9011-7c90efad5a33 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:11.695593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf1577-bbe8-4587-afbd-2dd16bf25d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:11.696524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf1577-bbe8-4587-afbd-2dd16bf25d37 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:11.697295 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf1577-bbe8-4587-afbd-2dd16bf25d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:11.697606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf1577-bbe8-4587-afbd-2dd16bf25d37 None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:11.699031 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf1577-bbe8-4587-afbd-2dd16bf25d37 None admin] RBAC: Authorizing `identity:get_project(project_id=79d2a60c71124efc90e0d0b445517d55)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:11.700883 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf1577-bbe8-4587-afbd-2dd16bf25d37 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:11.703256 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 71/134] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:11 2022] GET /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:11.712078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf1577-bbe8-4587-afbd-2dd16bf25d37 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:11.713377 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b3f3f-d9f1-47b9-b475-953adccfcb62 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:11.809089 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7f635-9888-48be-8e38-35cdc5c4634c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:11.811202 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7f635-9888-48be-8e38-35cdc5c4634c None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:11.813026 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7f635-9888-48be-8e38-35cdc5c4634c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:11.813619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7f635-9888-48be-8e38-35cdc5c4634c None admin] PATH_INFO: `/v3/projects/5789528db5bf4c14a1dc84d2cefd203f` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:11.816992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7f635-9888-48be-8e38-35cdc5c4634c None admin] RBAC: Authorizing `identity:get_project(project_id=5789528db5bf4c14a1dc84d2cefd203f)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:11.818817 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7f635-9888-48be-8e38-35cdc5c4634c None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:11.827874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 64/135] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:11 2022] GET /identity/v3/projects/5789528db5bf4c14a1dc84d2cefd203f => generated 312 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:11.829662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7f635-9888-48be-8e38-35cdc5c4634c None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:11.844611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:11.893473 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:11.902192 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf1577-bbe8-4587-afbd-2dd16bf25d37 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:11.951860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190eff7-dc1f-4c2d-bda3-f748fa636c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:11.952823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190eff7-dc1f-4c2d-bda3-f748fa636c96 None admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:11.953113 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190eff7-dc1f-4c2d-bda3-f748fa636c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:11.953387 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190eff7-dc1f-4c2d-bda3-f748fa636c96 None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55/users/ee9d005c534c4b0da3bb98afdb8596a9/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:11.955422 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190eff7-dc1f-4c2d-bda3-f748fa636c96 None admin] RBAC: Authorizing `identity:create_grant(project_id=79d2a60c71124efc90e0d0b445517d55, user_id=ee9d005c534c4b0da3bb98afdb8596a9, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:11.957692 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190eff7-dc1f-4c2d-bda3-f748fa636c96 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:12.001552 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7f635-9888-48be-8e38-35cdc5c4634c None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:12.035665 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 72/136] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:03:11 2022] PUT /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55/users/ee9d005c534c4b0da3bb98afdb8596a9/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:03:12.037127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74868b-fdb1-4439-96a2-c3d3dd57b043 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:12.037425 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74868b-fdb1-4439-96a2-c3d3dd57b043 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:12.037707 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74868b-fdb1-4439-96a2-c3d3dd57b043 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:12.040062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 73/137] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:12.046010 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65884f0-9b94-4af2-ac8c-8295b142c30d None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:12.046299 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65884f0-9b94-4af2-ac8c-8295b142c30d None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:12.052507 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65884f0-9b94-4af2-ac8c-8295b142c30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:12.062766 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007c627-e61b-421e-9483-2d2a4930dfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:12.064394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007c627-e61b-421e-9483-2d2a4930dfa9 None admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:12.064714 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007c627-e61b-421e-9483-2d2a4930dfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:12.067296 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007c627-e61b-421e-9483-2d2a4930dfa9 None admin] PATH_INFO: `/v3/projects/5789528db5bf4c14a1dc84d2cefd203f/users/ee9d005c534c4b0da3bb98afdb8596a9/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:12.071394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007c627-e61b-421e-9483-2d2a4930dfa9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5789528db5bf4c14a1dc84d2cefd203f, user_id=ee9d005c534c4b0da3bb98afdb8596a9, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:12.076552 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007c627-e61b-421e-9483-2d2a4930dfa9 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:12.084205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-c65884f0-9b94-4af2-ac8c-8295b142c30d None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:12.086318 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-c65884f0-9b94-4af2-ac8c-8295b142c30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:12.171634 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 74/138] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 23 09:03:12.189567 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 65/139] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:03:11 2022] PUT /identity/v3/projects/5789528db5bf4c14a1dc84d2cefd203f/users/ee9d005c534c4b0da3bb98afdb8596a9/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:03:12.223727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65884f0-9b94-4af2-ac8c-8295b142c30d None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:12.309124 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:12.359763 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65884f0-9b94-4af2-ac8c-8295b142c30d None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:12.402136 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4661ae6-d5da-4e87-a436-dd449a4df1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:12.402961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4661ae6-d5da-4e87-a436-dd449a4df1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:12.403789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4661ae6-d5da-4e87-a436-dd449a4df1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:12.404075 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4661ae6-d5da-4e87-a436-dd449a4df1e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:12.405278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4661ae6-d5da-4e87-a436-dd449a4df1e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:12.406753 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4661ae6-d5da-4e87-a436-dd449a4df1e2 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:12.408818 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 75/140] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:03:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:12.414491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007c627-e61b-421e-9483-2d2a4930dfa9 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:12.524925 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:12.540769 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007c627-e61b-421e-9483-2d2a4930dfa9 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:12.596014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f177d-a210-4f93-bb93-6691488f645a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:12.596700 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f177d-a210-4f93-bb93-6691488f645a None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:12.596998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f177d-a210-4f93-bb93-6691488f645a None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:12.597291 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f177d-a210-4f93-bb93-6691488f645a None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:12.598033 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f177d-a210-4f93-bb93-6691488f645a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:12.599659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f177d-a210-4f93-bb93-6691488f645a None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:12.713572 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 66/141] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jun 23 09:03:12 2022] POST /identity/v3/users => generated 283 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:03:12.979844 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e2652-87ae-4891-85b7-e48026271d13 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:12.980327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e2652-87ae-4891-85b7-e48026271d13 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:12.980327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e2652-87ae-4891-85b7-e48026271d13 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:12.982247 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 76/142] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:12.997731 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c964ef-1c87-4704-86e1-501a6414ce5f None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:12.998083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c964ef-1c87-4704-86e1-501a6414ce5f None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:12.998316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c964ef-1c87-4704-86e1-501a6414ce5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:13.018888 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-83c964ef-1c87-4704-86e1-501a6414ce5f None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:13.020134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-83c964ef-1c87-4704-86e1-501a6414ce5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:13.093472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 67/143] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:13.141420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e2652-87ae-4891-85b7-e48026271d13 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:13.283807 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e2652-87ae-4891-85b7-e48026271d13 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:13.335105 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:13.348551 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49c039-ba3c-4c7e-818c-73eba9575722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:13.349372 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49c039-ba3c-4c7e-818c-73eba9575722 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:13.354926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49c039-ba3c-4c7e-818c-73eba9575722 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:13.354926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49c039-ba3c-4c7e-818c-73eba9575722 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:13.366543 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db49c039-ba3c-4c7e-818c-73eba9575722 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:13.375099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-db49c039-ba3c-4c7e-818c-73eba9575722 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 09:03:13.375099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:13.375099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:13.375099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:13.375099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:13.375099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:13.375099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:13.375099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:13.375099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:13.375099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:13.375099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:13.375099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:13.375099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:13.375099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:13.375099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:13.375099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:13.376004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:13.376004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:13.376004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:13.376004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:13.376004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:13.376004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:13.376004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:13.376004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:13.376004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:13.376004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:13.376004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:13.376004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:13.376004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:13.376004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:13.376004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:13.376004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:13.376828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:13.376828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:13.376828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:13.376828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:13.376828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:13.376828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:13.376828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:13.376828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:13.376828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:13.376828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:13.376828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 09:03:13.376828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:13.376828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 77/144] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:13.383570 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c964ef-1c87-4704-86e1-501a6414ce5f None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:13.478353 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:13.514809 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c964ef-1c87-4704-86e1-501a6414ce5f None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:13.558497 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4c6d6-b876-40f6-bd58-92b47b523f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:13.559245 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4c6d6-b876-40f6-bd58-92b47b523f55 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:13.559578 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4c6d6-b876-40f6-bd58-92b47b523f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:13.559939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4c6d6-b876-40f6-bd58-92b47b523f55 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:13.560634 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4c6d6-b876-40f6-bd58-92b47b523f55 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:13.561619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4c6d6-b876-40f6-bd58-92b47b523f55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:13.563532 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4c6d6-b876-40f6-bd58-92b47b523f55 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:13.579472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 68/145] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:13.585080 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49c039-ba3c-4c7e-818c-73eba9575722 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:13.786104 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49c039-ba3c-4c7e-818c-73eba9575722 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:13.821983 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c28cf0-e210-4408-b250-f60825adb721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:13.822802 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c28cf0-e210-4408-b250-f60825adb721 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:13.823121 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c28cf0-e210-4408-b250-f60825adb721 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:13.823409 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c28cf0-e210-4408-b250-f60825adb721 None admin] PATH_INFO: `/v3/users/ca520b6a862c447987cd8f3b34dc58e7` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:13.824724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c28cf0-e210-4408-b250-f60825adb721 None admin] RBAC: Authorizing `identity:get_user(user_id=ca520b6a862c447987cd8f3b34dc58e7)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:13.826789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c28cf0-e210-4408-b250-f60825adb721 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:13.828482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 78/146] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:13 2022] GET /identity/v3/users/ca520b6a862c447987cd8f3b34dc58e7 => generated 289 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:13.834249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4c6d6-b876-40f6-bd58-92b47b523f55 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:14.006539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4c6d6-b876-40f6-bd58-92b47b523f55 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:14.053802 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710aee02-b546-4be5-9b5c-1db8a7fd15b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:14.054700 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710aee02-b546-4be5-9b5c-1db8a7fd15b3 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:14.055020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710aee02-b546-4be5-9b5c-1db8a7fd15b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:14.055304 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710aee02-b546-4be5-9b5c-1db8a7fd15b3 None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:14.059930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710aee02-b546-4be5-9b5c-1db8a7fd15b3 None admin] RBAC: Authorizing `identity:get_project(project_id=79d2a60c71124efc90e0d0b445517d55)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:14.061452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710aee02-b546-4be5-9b5c-1db8a7fd15b3 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:14.063614 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 69/147] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:13 2022] GET /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:14.069168 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c28cf0-e210-4408-b250-f60825adb721 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:14.112946 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb41fe-c421-41e7-9efd-b79636688a2e None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:14.113239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb41fe-c421-41e7-9efd-b79636688a2e None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:14.116329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb41fe-c421-41e7-9efd-b79636688a2e None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:14.116329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 70/148] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:14.124002 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70a30f-430a-494c-8b3a-d04975d5928d None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:14.124330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70a30f-430a-494c-8b3a-d04975d5928d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:14.124562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70a30f-430a-494c-8b3a-d04975d5928d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:14.136188 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:14.159989 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-ed70a30f-430a-494c-8b3a-d04975d5928d None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:14.161206 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-ed70a30f-430a-494c-8b3a-d04975d5928d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:14.187149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:14.279023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c28cf0-e210-4408-b250-f60825adb721 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:14.329223 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 71/149] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:03:14.343424 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b75de-35a6-4391-a598-5e76aaf39d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:14.344668 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b75de-35a6-4391-a598-5e76aaf39d39 None admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:14.345191 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b75de-35a6-4391-a598-5e76aaf39d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:14.345684 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b75de-35a6-4391-a598-5e76aaf39d39 None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55/users/ca520b6a862c447987cd8f3b34dc58e7/roles/a6f4585744cf42e6b971891968972799` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:14.348746 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b75de-35a6-4391-a598-5e76aaf39d39 None admin] RBAC: Authorizing `identity:create_grant(project_id=79d2a60c71124efc90e0d0b445517d55, user_id=ca520b6a862c447987cd8f3b34dc58e7, role_id=a6f4585744cf42e6b971891968972799)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:14.350426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b75de-35a6-4391-a598-5e76aaf39d39 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:14.371408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70a30f-430a-494c-8b3a-d04975d5928d None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:14.430010 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 79/150] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:03:14 2022] PUT /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55/users/ca520b6a862c447987cd8f3b34dc58e7/roles/a6f4585744cf42e6b971891968972799 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:03:14.515268 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70a30f-430a-494c-8b3a-d04975d5928d None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:14.551523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:14.557241 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c3873-96c5-4bd7-89b2-d8e61e700037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:14.558365 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c3873-96c5-4bd7-89b2-d8e61e700037 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:14.558887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c3873-96c5-4bd7-89b2-d8e61e700037 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:14.559469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c3873-96c5-4bd7-89b2-d8e61e700037 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:14.576285 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c3873-96c5-4bd7-89b2-d8e61e700037 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:14.585076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-9d7c3873-96c5-4bd7-89b2-d8e61e700037 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:14.585076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:14.585076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:14.585076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:14.585076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:14.585076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:14.585076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:14.585076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:14.585076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:14.585076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:14.585076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:14.585076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:14.585076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:14.585076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:14.585076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:14.585076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:14.586000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:14.586000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:14.586000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:14.586000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:14.586000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:14.586000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:14.586000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:14.586000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:14.586000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:14.586000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:14.586000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:14.586000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:14.586000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:14.586000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:14.586000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:14.586000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:14.586814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:14.586814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:14.586814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:14.586814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:14.586814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:14.586814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:14.586814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:14.586814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:14.586814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:14.586814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:14.586814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:14.586814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:14.589172 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 72/151] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:03:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:03:14.595276 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b75de-35a6-4391-a598-5e76aaf39d39 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:14.697362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d26ad1-682f-4738-a895-8284760c0ec3 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:14.697863 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d26ad1-682f-4738-a895-8284760c0ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:14.698289 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d26ad1-682f-4738-a895-8284760c0ec3 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:14.699958 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 73/152] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:14.705489 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa00d78-7a8e-4619-8f2a-1a0c6a68a52a None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:14.705985 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa00d78-7a8e-4619-8f2a-1a0c6a68a52a None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:14.706413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa00d78-7a8e-4619-8f2a-1a0c6a68a52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:14.743542 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b75de-35a6-4391-a598-5e76aaf39d39 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:14.752234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-1fa00d78-7a8e-4619-8f2a-1a0c6a68a52a None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:14.753823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-1fa00d78-7a8e-4619-8f2a-1a0c6a68a52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:14.760237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:14.782667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20f2a0-0fcb-4ec2-aee9-673a9c2c99d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:14.783698 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20f2a0-0fcb-4ec2-aee9-673a9c2c99d4 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:14.784227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20f2a0-0fcb-4ec2-aee9-673a9c2c99d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:14.784708 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20f2a0-0fcb-4ec2-aee9-673a9c2c99d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:14.785649 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20f2a0-0fcb-4ec2-aee9-673a9c2c99d4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:14.786370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20f2a0-0fcb-4ec2-aee9-673a9c2c99d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:14.788105 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20f2a0-0fcb-4ec2-aee9-673a9c2c99d4 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:14.798967 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 80/153] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:03:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:14.806993 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20f2a0-0fcb-4ec2-aee9-673a9c2c99d4 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:14.902176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 74/154] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:03:14.938218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa00d78-7a8e-4619-8f2a-1a0c6a68a52a None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:14.945498 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20f2a0-0fcb-4ec2-aee9-673a9c2c99d4 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:14.996496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687c936-31c1-4017-8530-d80fd0f9de5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:14.997697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687c936-31c1-4017-8530-d80fd0f9de5e None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:14.998260 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687c936-31c1-4017-8530-d80fd0f9de5e None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:14.998728 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687c936-31c1-4017-8530-d80fd0f9de5e None admin] PATH_INFO: `/v3/users/ee9d005c534c4b0da3bb98afdb8596a9` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:15.000232 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687c936-31c1-4017-8530-d80fd0f9de5e None admin] RBAC: Authorizing `identity:get_user(user_id=ee9d005c534c4b0da3bb98afdb8596a9)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:15.001921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687c936-31c1-4017-8530-d80fd0f9de5e None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:15.003910 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 81/155] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:14 2022] GET /identity/v3/users/ee9d005c534c4b0da3bb98afdb8596a9 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:15.009562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687c936-31c1-4017-8530-d80fd0f9de5e None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:15.076562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:15.156824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687c936-31c1-4017-8530-d80fd0f9de5e None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:15.182698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa00d78-7a8e-4619-8f2a-1a0c6a68a52a None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:15.229817 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:15.245257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f7d21-1ef5-4b32-a117-a5ff66d0e75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:15.246386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f7d21-1ef5-4b32-a117-a5ff66d0e75f None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:15.246994 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f7d21-1ef5-4b32-a117-a5ff66d0e75f None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:15.247466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f7d21-1ef5-4b32-a117-a5ff66d0e75f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:15.252458 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b4ea6-30bf-4495-8ac4-8981d52b3309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:15.255608 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b4ea6-30bf-4495-8ac4-8981d52b3309 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:15.255906 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b4ea6-30bf-4495-8ac4-8981d52b3309 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:15.256193 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b4ea6-30bf-4495-8ac4-8981d52b3309 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:15.274152 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b4ea6-30bf-4495-8ac4-8981d52b3309 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:15.284184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-212b4ea6-30bf-4495-8ac4-8981d52b3309 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:03:15.284184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:15.284184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:15.284184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:15.284184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:15.284184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:15.284184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:15.284184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:15.284184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:15.284184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:15.284184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:15.284184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:15.284184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:15.284184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:15.284184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:15.284184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:15.285187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:15.285187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:15.285187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:15.285187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:15.285187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:15.285187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:15.285187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:15.285187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:15.285187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:15.285187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:15.285187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:15.285187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:15.285187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:15.285187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:15.285187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:15.285187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:15.286008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:15.286008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:15.286008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:15.286008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:15.286008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:15.286008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:15.286008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:15.286008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:15.286008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:15.286008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:15.286008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:03:15.286008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:15.286008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 75/156] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:03:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:03:15.291109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b4ea6-30bf-4495-8ac4-8981d52b3309 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:15.373980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f7d21-1ef5-4b32-a117-a5ff66d0e75f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:15.375511 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f7d21-1ef5-4b32-a117-a5ff66d0e75f None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:15.425308 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b4ea6-30bf-4495-8ac4-8981d52b3309 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:15.428882 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:15.463337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b615084-57fb-4e8d-93c9-44077f2713c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:15.464371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b615084-57fb-4e8d-93c9-44077f2713c3 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:15.464954 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b615084-57fb-4e8d-93c9-44077f2713c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:15.465972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b615084-57fb-4e8d-93c9-44077f2713c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:15.467292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b615084-57fb-4e8d-93c9-44077f2713c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:15.467808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b615084-57fb-4e8d-93c9-44077f2713c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:15.469394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b615084-57fb-4e8d-93c9-44077f2713c3 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:15.479015 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 76/157] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:15.480610 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d48e78a-077e-4f58-bed5-e7a343258808 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:15.481124 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d48e78a-077e-4f58-bed5-e7a343258808 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:15.482121 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d48e78a-077e-4f58-bed5-e7a343258808 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:15.483967 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 77/158] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:15.485065 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d48e78a-077e-4f58-bed5-e7a343258808 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:15.504217 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 82/159] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 09:03:15.509641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ec654-1434-45dd-a29c-49a24389702b None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:15.510102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ec654-1434-45dd-a29c-49a24389702b None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:15.510397 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ec654-1434-45dd-a29c-49a24389702b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:15.535948 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-d20ec654-1434-45dd-a29c-49a24389702b None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:15.537167 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d20ec654-1434-45dd-a29c-49a24389702b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:15.544662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:15.603769 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d48e78a-077e-4f58-bed5-e7a343258808 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:15.640335 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9aa2b-8a2b-4bb6-9eae-964976ce8154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:15.641577 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9aa2b-8a2b-4bb6-9eae-964976ce8154 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:15.642123 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9aa2b-8a2b-4bb6-9eae-964976ce8154 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:15.642607 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9aa2b-8a2b-4bb6-9eae-964976ce8154 None admin] PATH_INFO: `/v3/users/261a7891b0d244a3a8e71c73d6ba80b6` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:15.644070 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b9aa2b-8a2b-4bb6-9eae-964976ce8154 None admin] RBAC: Authorizing `identity:get_user(user_id=261a7891b0d244a3a8e71c73d6ba80b6)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:15.646020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b9aa2b-8a2b-4bb6-9eae-964976ce8154 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:15.646499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 83/160] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:15.647465 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ec654-1434-45dd-a29c-49a24389702b None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:15.648406 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 78/161] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:15 2022] GET /identity/v3/users/261a7891b0d244a3a8e71c73d6ba80b6 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:15.654195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9aa2b-8a2b-4bb6-9eae-964976ce8154 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:15.751789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:15.770124 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9aa2b-8a2b-4bb6-9eae-964976ce8154 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:15.780571 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:15.802810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c87820-8bc8-45dc-b862-a2a647234f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:15.803859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c87820-8bc8-45dc-b862-a2a647234f18 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:15.804341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c87820-8bc8-45dc-b862-a2a647234f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:15.804814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c87820-8bc8-45dc-b862-a2a647234f18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:15.809314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ec654-1434-45dd-a29c-49a24389702b None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:15.881516 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c87820-8bc8-45dc-b862-a2a647234f18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:15.883437 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c87820-8bc8-45dc-b862-a2a647234f18 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:15.944244 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb81a6d-6fb9-41e6-a910-5b1d9d45a805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:15.945166 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb81a6d-6fb9-41e6-a910-5b1d9d45a805 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:15.945431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb81a6d-6fb9-41e6-a910-5b1d9d45a805 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:15.945714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb81a6d-6fb9-41e6-a910-5b1d9d45a805 None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:15.946950 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb81a6d-6fb9-41e6-a910-5b1d9d45a805 None admin] RBAC: Authorizing `identity:get_project(project_id=79d2a60c71124efc90e0d0b445517d55)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:15.948434 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb81a6d-6fb9-41e6-a910-5b1d9d45a805 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:15.950997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 84/162] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:15 2022] GET /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55 => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:15.951666 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb81a6d-6fb9-41e6-a910-5b1d9d45a805 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:16.003759 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 79/163] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:03:16.007664 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c87820-8bc8-45dc-b862-a2a647234f18 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:16.036361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:16.165045 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb81a6d-6fb9-41e6-a910-5b1d9d45a805 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:16.199363 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c87820-8bc8-45dc-b862-a2a647234f18 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:16.233577 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c017c-5353-4784-802f-cbdb5ad14325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:16.234552 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c017c-5353-4784-802f-cbdb5ad14325 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:16.234945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c017c-5353-4784-802f-cbdb5ad14325 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:16.235329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c017c-5353-4784-802f-cbdb5ad14325 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:16.254385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c017c-5353-4784-802f-cbdb5ad14325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:16.264504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-ee6c017c-5353-4784-802f-cbdb5ad14325 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:16.264504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:16.264504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:16.264504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:16.264504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:16.264504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:16.264504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:16.264504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:16.264504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:16.264504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:16.264504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:16.264504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:16.264504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:16.264504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:16.264504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:16.264504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:16.265598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:16.265598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:16.265598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:16.265598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:16.265598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:16.265598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:16.265598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:16.265598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:16.265598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:16.265598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:16.265598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:16.265598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:16.265598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:16.265598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:16.265598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:16.265598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:16.266620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:16.266620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:16.266620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:16.266620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:16.266620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:16.266620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:16.266620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:16.266620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:16.266620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:16.266620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:16.266620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:16.266620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:16.268681 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 85/164] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:16.269677 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c017c-5353-4784-802f-cbdb5ad14325 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:16.271240 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133dc5e-ea58-42f2-be04-b7653c795545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:16.272232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133dc5e-ea58-42f2-be04-b7653c795545 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:16.272723 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133dc5e-ea58-42f2-be04-b7653c795545 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:16.273196 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:16.274301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133dc5e-ea58-42f2-be04-b7653c795545 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:16.275388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133dc5e-ea58-42f2-be04-b7653c795545 None admin] RBAC: Adding query filter params (role.id=478b327fd22845cc98a2fafe821b1e85, scope.project.id=79d2a60c71124efc90e0d0b445517d55, user.id=ee9d005c534c4b0da3bb98afdb8596a9) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:16.275925 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133dc5e-ea58-42f2-be04-b7653c795545 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:16.277581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133dc5e-ea58-42f2-be04-b7653c795545 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:16.284875 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 80/165] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:03:16 2022] GET /identity/v3/role_assignments?user.id=ee9d005c534c4b0da3bb98afdb8596a9&scope.project.id=79d2a60c71124efc90e0d0b445517d55&role.id=478b327fd22845cc98a2fafe821b1e85 => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:16.286582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a3ee8-9d07-4b0a-87f1-92f635e0b1e1 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:16.287069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a3ee8-9d07-4b0a-87f1-92f635e0b1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:16.287554 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a3ee8-9d07-4b0a-87f1-92f635e0b1e1 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:16.289020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 81/166] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:16.290735 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a3ee8-9d07-4b0a-87f1-92f635e0b1e1 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:16.376147 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:16.403972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a3ee8-9d07-4b0a-87f1-92f635e0b1e1 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:16.446088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50244753-268e-409a-9bd2-b301ba2ece50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:16.447170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50244753-268e-409a-9bd2-b301ba2ece50 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:16.447665 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50244753-268e-409a-9bd2-b301ba2ece50 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:16.448184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50244753-268e-409a-9bd2-b301ba2ece50 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:16.449209 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50244753-268e-409a-9bd2-b301ba2ece50 None admin] RBAC: Adding query filter params (name=member) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:16.449752 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50244753-268e-409a-9bd2-b301ba2ece50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:16.451703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50244753-268e-409a-9bd2-b301ba2ece50 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:16.462930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 82/167] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:16.464829 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48087f2-3037-4add-af16-6be60b0a6a52 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:16.465347 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48087f2-3037-4add-af16-6be60b0a6a52 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:16.466626 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48087f2-3037-4add-af16-6be60b0a6a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:16.485492 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-d48087f2-3037-4add-af16-6be60b0a6a52 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:16.487413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-d48087f2-3037-4add-af16-6be60b0a6a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:16.497359 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c017c-5353-4784-802f-cbdb5ad14325 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:16.573095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 83/168] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:16.575882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48087f2-3037-4add-af16-6be60b0a6a52 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:16.587896 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b203ddf-3b92-45a5-9e57-1933244d447c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:16.588787 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b203ddf-3b92-45a5-9e57-1933244d447c None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:16.589072 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b203ddf-3b92-45a5-9e57-1933244d447c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:16.589350 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b203ddf-3b92-45a5-9e57-1933244d447c None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:16.593263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b203ddf-3b92-45a5-9e57-1933244d447c None admin] RBAC: Authorizing `identity:get_project(project_id=79d2a60c71124efc90e0d0b445517d55)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:16.594858 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b203ddf-3b92-45a5-9e57-1933244d447c None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:16.596900 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 86/169] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:16 2022] GET /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55 => generated 298 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:16.605270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b203ddf-3b92-45a5-9e57-1933244d447c None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:16.657541 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:16.698212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48087f2-3037-4add-af16-6be60b0a6a52 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:16.742138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b849c-15aa-4295-a33b-e33a2680a45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:16.743213 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b849c-15aa-4295-a33b-e33a2680a45d None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:16.744087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b849c-15aa-4295-a33b-e33a2680a45d None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:16.744441 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b849c-15aa-4295-a33b-e33a2680a45d None admin] PATH_INFO: `/v3/users/ee9d005c534c4b0da3bb98afdb8596a9` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:16.745877 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6b849c-15aa-4295-a33b-e33a2680a45d None admin] RBAC: Authorizing `identity:get_user(user_id=ee9d005c534c4b0da3bb98afdb8596a9)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:16.747625 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6b849c-15aa-4295-a33b-e33a2680a45d None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:16.749608 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 84/170] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:16 2022] GET /identity/v3/users/ee9d005c534c4b0da3bb98afdb8596a9 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:16.750446 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b849c-15aa-4295-a33b-e33a2680a45d None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:16.765237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:16.817618 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b203ddf-3b92-45a5-9e57-1933244d447c None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:16.873445 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b849c-15aa-4295-a33b-e33a2680a45d None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:16.885864 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061ea60-d7ed-43f3-b081-290b6123a4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:16.887010 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061ea60-d7ed-43f3-b081-290b6123a4e7 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:16.887549 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061ea60-d7ed-43f3-b081-290b6123a4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:16.888034 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061ea60-d7ed-43f3-b081-290b6123a4e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:16.889187 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061ea60-d7ed-43f3-b081-290b6123a4e7 None admin] RBAC: Adding query filter params (role.id=9cd080b50c6f4ee8a54bf9ef1655bd83, scope.project.id=79d2a60c71124efc90e0d0b445517d55, user.id=261a7891b0d244a3a8e71c73d6ba80b6) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:16.889779 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061ea60-d7ed-43f3-b081-290b6123a4e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:16.893057 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061ea60-d7ed-43f3-b081-290b6123a4e7 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:16.902162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 87/171] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:03:16 2022] GET /identity/v3/role_assignments?user.id=261a7891b0d244a3a8e71c73d6ba80b6&scope.project.id=79d2a60c71124efc90e0d0b445517d55&role.id=9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:16.904087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061ea60-d7ed-43f3-b081-290b6123a4e7 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:16.915044 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42f89f-3711-43ee-9841-403818bec221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:16.915897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42f89f-3711-43ee-9841-403818bec221 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:16.916228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42f89f-3711-43ee-9841-403818bec221 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:16.916505 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42f89f-3711-43ee-9841-403818bec221 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:16.919869 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:16.932327 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42f89f-3711-43ee-9841-403818bec221 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:16.942467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-5e42f89f-3711-43ee-9841-403818bec221 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:16.942467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:16.942467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:16.942467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:16.942467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:16.942467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:16.942467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:16.942467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:16.942467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:16.942467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:16.942467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:16.942467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:16.942467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:16.942467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:16.942467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:16.942467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:16.943522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:16.943522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:16.943522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:16.943522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:16.943522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:16.943522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:16.943522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:16.943522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:16.943522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:16.943522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:16.943522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:16.943522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:16.943522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:16.943522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:16.943522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:16.943522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:16.945132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:16.945132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:16.945132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:16.945132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:16.945132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:16.945132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:16.945132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:16.945132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:16.945132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:16.945132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:16.945132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:16.945132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:16.945132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 85/172] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:16.950661 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42f89f-3711-43ee-9841-403818bec221 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:17.075755 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42f89f-3711-43ee-9841-403818bec221 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:17.118015 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c162c-bbd5-4d51-946f-dfad1ccd5c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:17.118744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c162c-bbd5-4d51-946f-dfad1ccd5c48 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:17.119129 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c162c-bbd5-4d51-946f-dfad1ccd5c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:17.119408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c162c-bbd5-4d51-946f-dfad1ccd5c48 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:17.120354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c162c-bbd5-4d51-946f-dfad1ccd5c48 None admin] RBAC: Adding query filter params (name=member) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:17.120668 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c162c-bbd5-4d51-946f-dfad1ccd5c48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:17.122607 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c162c-bbd5-4d51-946f-dfad1ccd5c48 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:17.132876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 86/173] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:17.138804 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c162c-bbd5-4d51-946f-dfad1ccd5c48 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:17.154802 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061ea60-d7ed-43f3-b081-290b6123a4e7 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:17.215202 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:17.251754 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873fd00-5875-4be4-9e91-e99ab1873cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:17.252417 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873fd00-5875-4be4-9e91-e99ab1873cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:17.253661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873fd00-5875-4be4-9e91-e99ab1873cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:17.253661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873fd00-5875-4be4-9e91-e99ab1873cd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:17.254502 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c162c-bbd5-4d51-946f-dfad1ccd5c48 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:17.261272 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:17.291991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f71dd6-e2be-4e45-aff2-04f1d1779f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:17.292927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f71dd6-e2be-4e45-aff2-04f1d1779f5e None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:17.293213 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f71dd6-e2be-4e45-aff2-04f1d1779f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:17.294462 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f71dd6-e2be-4e45-aff2-04f1d1779f5e None admin] PATH_INFO: `/v3/users/ee9d005c534c4b0da3bb98afdb8596a9` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:17.295745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f71dd6-e2be-4e45-aff2-04f1d1779f5e None admin] RBAC: Authorizing `identity:get_user(user_id=ee9d005c534c4b0da3bb98afdb8596a9)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:17.297255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f71dd6-e2be-4e45-aff2-04f1d1779f5e None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:17.299301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 87/174] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:17 2022] GET /identity/v3/users/ee9d005c534c4b0da3bb98afdb8596a9 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:17.301378 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9e388-41db-4872-bd8f-658d2df5764c None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:17.301665 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9e388-41db-4872-bd8f-658d2df5764c None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:17.301939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9e388-41db-4872-bd8f-658d2df5764c None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:17.307672 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 88/175] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:17.308878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9e388-41db-4872-bd8f-658d2df5764c None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:17.422601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9e388-41db-4872-bd8f-658d2df5764c None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:17.435778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873fd00-5875-4be4-9e91-e99ab1873cd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:17.437598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873fd00-5875-4be4-9e91-e99ab1873cd7 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:17.454284 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:17.458169 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25832e1-1748-4134-9088-b5b8228c11ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:17.458830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25832e1-1748-4134-9088-b5b8228c11ad None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:17.459208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25832e1-1748-4134-9088-b5b8228c11ad None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:17.459486 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25832e1-1748-4134-9088-b5b8228c11ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:17.535467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25832e1-1748-4134-9088-b5b8228c11ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:17.536946 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25832e1-1748-4134-9088-b5b8228c11ad None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:17.649853 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 89/176] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 09:03:17.655463 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598421df-65db-4dfc-bb32-e9bf399c628c None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:17.655780 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598421df-65db-4dfc-bb32-e9bf399c628c None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:17.656067 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598421df-65db-4dfc-bb32-e9bf399c628c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:17.675571 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-598421df-65db-4dfc-bb32-e9bf399c628c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:17.676808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-598421df-65db-4dfc-bb32-e9bf399c628c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:17.716056 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:17.754897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 90/177] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:17.756170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598421df-65db-4dfc-bb32-e9bf399c628c None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:17.758256 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 88/178] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 855 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:03:17.764403 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873fd00-5875-4be4-9e91-e99ab1873cd7 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:17.883067 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598421df-65db-4dfc-bb32-e9bf399c628c None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:17.898063 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:17.922462 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706533e3-69ea-45a7-9f12-25b3836574d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:17.923376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706533e3-69ea-45a7-9f12-25b3836574d8 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:17.923665 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706533e3-69ea-45a7-9f12-25b3836574d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:17.924070 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706533e3-69ea-45a7-9f12-25b3836574d8 None admin] PATH_INFO: `/v3/projects/5789528db5bf4c14a1dc84d2cefd203f` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:17.925402 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706533e3-69ea-45a7-9f12-25b3836574d8 None admin] RBAC: Authorizing `identity:get_project(project_id=5789528db5bf4c14a1dc84d2cefd203f)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:17.926905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706533e3-69ea-45a7-9f12-25b3836574d8 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:17.928801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 91/179] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:17 2022] GET /identity/v3/projects/5789528db5bf4c14a1dc84d2cefd203f => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:17.930017 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706533e3-69ea-45a7-9f12-25b3836574d8 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:17.959674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:18.056756 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873fd00-5875-4be4-9e91-e99ab1873cd7 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:18.094825 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46722cfd-a1ba-4209-ad7f-d4a4c03dd625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:18.095917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46722cfd-a1ba-4209-ad7f-d4a4c03dd625 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:18.096464 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46722cfd-a1ba-4209-ad7f-d4a4c03dd625 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:18.096963 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46722cfd-a1ba-4209-ad7f-d4a4c03dd625 None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:18.098467 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46722cfd-a1ba-4209-ad7f-d4a4c03dd625 None admin] RBAC: Authorizing `identity:get_project(project_id=79d2a60c71124efc90e0d0b445517d55)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:18.100708 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46722cfd-a1ba-4209-ad7f-d4a4c03dd625 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:18.103078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 89/180] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:17 2022] GET /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55 => generated 298 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:18.103770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46722cfd-a1ba-4209-ad7f-d4a4c03dd625 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:18.117547 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706533e3-69ea-45a7-9f12-25b3836574d8 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:18.182893 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00dc43-db77-41e3-9244-dcda63e530ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:18.183655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00dc43-db77-41e3-9244-dcda63e530ea None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:18.183941 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00dc43-db77-41e3-9244-dcda63e530ea None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:18.184278 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00dc43-db77-41e3-9244-dcda63e530ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:18.198134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00dc43-db77-41e3-9244-dcda63e530ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:18.208901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-5e00dc43-db77-41e3-9244-dcda63e530ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:03:18.208901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:18.208901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:18.208901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:18.208901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:18.208901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:18.208901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:18.208901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:18.208901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:18.208901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:18.208901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:18.208901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:18.208901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:18.208901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:18.208901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:18.208901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:18.209842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:18.209842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:18.209842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:18.209842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:18.209842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:18.209842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:18.209842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:18.209842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:18.209842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:18.209842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:18.209842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:18.209842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:18.209842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:18.209842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:18.209842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:18.209842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:18.210691 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:18.210691 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:18.210691 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:18.210691 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:18.210691 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:18.210691 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:18.210691 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:18.210691 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:18.210691 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:18.210691 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:18.210691 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:03:18.210691 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:18.210691 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 92/181] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:03:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:18.211418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00dc43-db77-41e3-9244-dcda63e530ea None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:18.230194 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46722cfd-a1ba-4209-ad7f-d4a4c03dd625 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:18.232036 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:18.266102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e873c1-9f52-417c-82c4-630ae5c22916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:18.266766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e873c1-9f52-417c-82c4-630ae5c22916 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:18.267094 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e873c1-9f52-417c-82c4-630ae5c22916 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:18.267372 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e873c1-9f52-417c-82c4-630ae5c22916 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:18.268210 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e873c1-9f52-417c-82c4-630ae5c22916 None admin] RBAC: Adding query filter params (role.id=53a3d65343394e7caafcc33f12cf138b, scope.project.id=5789528db5bf4c14a1dc84d2cefd203f, user.id=ee9d005c534c4b0da3bb98afdb8596a9) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:18.268528 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e873c1-9f52-417c-82c4-630ae5c22916 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:18.269992 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e873c1-9f52-417c-82c4-630ae5c22916 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:18.276447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 90/182] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:03:18 2022] GET /identity/v3/role_assignments?user.id=ee9d005c534c4b0da3bb98afdb8596a9&scope.project.id=5789528db5bf4c14a1dc84d2cefd203f&role.id=53a3d65343394e7caafcc33f12cf138b => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:18.277788 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20478e9c-6437-4ec8-9c08-02590e8f5f1e None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:18.278640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20478e9c-6437-4ec8-9c08-02590e8f5f1e None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:18.278945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20478e9c-6437-4ec8-9c08-02590e8f5f1e None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:18.280177 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 91/183] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:18.281283 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c05efc-12be-47ce-a9da-84bb3c545d21 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:18.281576 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c05efc-12be-47ce-a9da-84bb3c545d21 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:18.281855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c05efc-12be-47ce-a9da-84bb3c545d21 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:18.286960 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 92/184] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:18.286960 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c05efc-12be-47ce-a9da-84bb3c545d21 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:18.314400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:18.332931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00dc43-db77-41e3-9244-dcda63e530ea None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:18.372513 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453d97f-22fd-4ce7-ab55-94c3997190f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:18.373203 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453d97f-22fd-4ce7-ab55-94c3997190f7 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:18.373494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453d97f-22fd-4ce7-ab55-94c3997190f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:18.373777 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453d97f-22fd-4ce7-ab55-94c3997190f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:18.374636 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453d97f-22fd-4ce7-ab55-94c3997190f7 None admin] RBAC: Adding query filter params (role.id=53a3d65343394e7caafcc33f12cf138b, scope.project.id=79d2a60c71124efc90e0d0b445517d55, user.id=ee9d005c534c4b0da3bb98afdb8596a9) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:18.374982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453d97f-22fd-4ce7-ab55-94c3997190f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:18.376467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453d97f-22fd-4ce7-ab55-94c3997190f7 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:18.390660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 93/185] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:03:18 2022] GET /identity/v3/role_assignments?user.id=ee9d005c534c4b0da3bb98afdb8596a9&scope.project.id=79d2a60c71124efc90e0d0b445517d55&role.id=53a3d65343394e7caafcc33f12cf138b => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:18.390660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11df34-ae75-4394-8fc2-723aae67398f None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:18.390660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11df34-ae75-4394-8fc2-723aae67398f None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:18.390660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11df34-ae75-4394-8fc2-723aae67398f None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:18.390660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 94/186] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:18.390660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa288dc-eff9-44a8-b0f7-134ec92c552f None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:18.391270 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa288dc-eff9-44a8-b0f7-134ec92c552f None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:18.391270 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa288dc-eff9-44a8-b0f7-134ec92c552f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:18.409044 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-3fa288dc-eff9-44a8-b0f7-134ec92c552f None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:18.410531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-3fa288dc-eff9-44a8-b0f7-134ec92c552f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:18.413864 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c05efc-12be-47ce-a9da-84bb3c545d21 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:18.470703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09234e93-5c18-4dd1-98f4-d9b9fb148815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:18.471759 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09234e93-5c18-4dd1-98f4-d9b9fb148815 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:18.472237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09234e93-5c18-4dd1-98f4-d9b9fb148815 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:18.472793 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09234e93-5c18-4dd1-98f4-d9b9fb148815 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:18.474849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09234e93-5c18-4dd1-98f4-d9b9fb148815 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:18.475560 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09234e93-5c18-4dd1-98f4-d9b9fb148815 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:18.477024 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09234e93-5c18-4dd1-98f4-d9b9fb148815 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:18.488491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 93/187] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:18.490182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2da624-e26f-4dff-80fc-083cf7c3386c None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:18.490628 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2da624-e26f-4dff-80fc-083cf7c3386c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:18.491074 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2da624-e26f-4dff-80fc-083cf7c3386c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:18.510384 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 95/188] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:18.512049 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6355a9c-cf5c-4a89-88a3-9828570a757e None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:18.512049 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6355a9c-cf5c-4a89-88a3-9828570a757e None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:18.512420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6355a9c-cf5c-4a89-88a3-9828570a757e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:18.512980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-6b2da624-e26f-4dff-80fc-083cf7c3386c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:18.513901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-6b2da624-e26f-4dff-80fc-083cf7c3386c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:18.531727 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-f6355a9c-cf5c-4a89-88a3-9828570a757e None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:18.532911 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-f6355a9c-cf5c-4a89-88a3-9828570a757e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:18.541402 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:18.602895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 94/189] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:18.603794 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2da624-e26f-4dff-80fc-083cf7c3386c None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:18.629230 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 96/190] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:18.630083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6355a9c-cf5c-4a89-88a3-9828570a757e None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:18.641574 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:18.713020 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2da624-e26f-4dff-80fc-083cf7c3386c None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:18.735081 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6355a9c-cf5c-4a89-88a3-9828570a757e None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:18.746608 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572bfda-4581-40ed-85da-99b6e24468a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:18.747557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572bfda-4581-40ed-85da-99b6e24468a3 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:18.747847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572bfda-4581-40ed-85da-99b6e24468a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:18.748118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572bfda-4581-40ed-85da-99b6e24468a3 None admin] PATH_INFO: `/v3/users/ada63f4d813a4591a86492608c197e26` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:18.761498 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1572bfda-4581-40ed-85da-99b6e24468a3 None admin] RBAC: Authorizing `identity:get_user(user_id=ada63f4d813a4591a86492608c197e26)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:18.763520 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1572bfda-4581-40ed-85da-99b6e24468a3 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:18.765173 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 95/191] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:18 2022] GET /identity/v3/users/ada63f4d813a4591a86492608c197e26 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:18.765935 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572bfda-4581-40ed-85da-99b6e24468a3 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:18.768653 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-added045-a91d-4680-9344-4a056948c256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:18.769472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-added045-a91d-4680-9344-4a056948c256 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:18.769757 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-added045-a91d-4680-9344-4a056948c256 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:18.770033 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-added045-a91d-4680-9344-4a056948c256 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:18.780438 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-added045-a91d-4680-9344-4a056948c256 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:18.786126 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:18.788494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-added045-a91d-4680-9344-4a056948c256 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:18.788494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:18.788494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:18.788494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:18.788494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:18.788494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:18.788494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:18.788494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:18.788494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:18.788494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:18.788494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:18.788494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:18.788494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:18.788494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:18.788494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:18.788494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:18.789633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:18.789633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:18.789633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:18.789633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:18.789633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:18.789633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:18.789633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:18.789633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:18.789633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:18.789633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:18.789633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:18.789633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:18.789633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:18.789633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:18.789633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:18.789633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:18.790594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:18.790594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:18.790594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:18.790594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:18.790594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:18.790594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:18.790594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:18.790594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:18.790594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:18.790594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:18.790594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:18.790594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:18.790594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 97/192] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:03:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:18.790594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-added045-a91d-4680-9344-4a056948c256 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:18.858243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:18.867934 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572bfda-4581-40ed-85da-99b6e24468a3 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:18.891466 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-added045-a91d-4680-9344-4a056948c256 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:18.901420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b4d3e-cd81-4889-8f2c-30820bd5e038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:18.902208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b4d3e-cd81-4889-8f2c-30820bd5e038 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:18.902490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b4d3e-cd81-4889-8f2c-30820bd5e038 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:18.902766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b4d3e-cd81-4889-8f2c-30820bd5e038 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:18.913280 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b4d3e-cd81-4889-8f2c-30820bd5e038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:18.921008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-144b4d3e-cd81-4889-8f2c-30820bd5e038 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 09:03:18.921008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:18.921008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:18.921008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:18.921008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:18.921008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:18.921008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:18.921008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:18.921008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:18.921008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:18.921008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:18.921008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:18.921008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:18.921008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:18.921008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:18.921008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:18.921982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:18.921982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:18.921982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:18.921982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:18.921982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:18.921982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:18.921982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:18.921982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:18.921982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:18.921982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:18.921982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:18.921982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:18.921982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:18.921982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:18.921982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:18.921982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:18.922847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:18.922847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:18.922847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:18.922847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:18.922847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:18.922847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:18.922847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:18.922847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:18.922847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:18.922847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:18.922847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 09:03:18.922847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:18.922847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 96/193] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:18.923602 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b4d3e-cd81-4889-8f2c-30820bd5e038 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:18.929799 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a4783-20a7-48c7-a63b-4ee905b2ed2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:18.930599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a4783-20a7-48c7-a63b-4ee905b2ed2e None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:18.930887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a4783-20a7-48c7-a63b-4ee905b2ed2e None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:18.931200 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a4783-20a7-48c7-a63b-4ee905b2ed2e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:18.932409 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a4783-20a7-48c7-a63b-4ee905b2ed2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:18.933883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a4783-20a7-48c7-a63b-4ee905b2ed2e None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:18.935529 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 98/194] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:03:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:18.940073 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a4783-20a7-48c7-a63b-4ee905b2ed2e None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:19.002010 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:19.023744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b4d3e-cd81-4889-8f2c-30820bd5e038 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:19.037421 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a4783-20a7-48c7-a63b-4ee905b2ed2e None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:19.060097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024ce6b-21be-48ce-90f5-ca4d967e5d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:19.060747 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024ce6b-21be-48ce-90f5-ca4d967e5d86 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:19.061034 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024ce6b-21be-48ce-90f5-ca4d967e5d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:19.061315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024ce6b-21be-48ce-90f5-ca4d967e5d86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:19.070375 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d4262-cce2-4797-b194-44fe8d7bb6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:19.071087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d4262-cce2-4797-b194-44fe8d7bb6dc None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:19.071375 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d4262-cce2-4797-b194-44fe8d7bb6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:19.071652 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d4262-cce2-4797-b194-44fe8d7bb6dc None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:19.072400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993d4262-cce2-4797-b194-44fe8d7bb6dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:19.072712 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993d4262-cce2-4797-b194-44fe8d7bb6dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:19.074114 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993d4262-cce2-4797-b194-44fe8d7bb6dc None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:19.076802 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:19.083798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 99/195] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:03:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:19.088546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d4262-cce2-4797-b194-44fe8d7bb6dc None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:19.221925 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9024ce6b-21be-48ce-90f5-ca4d967e5d86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:19.223442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9024ce6b-21be-48ce-90f5-ca4d967e5d86 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:19.252148 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d4262-cce2-4797-b194-44fe8d7bb6dc None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:19.286906 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6671f1-9e6c-4179-b150-7d677db9b164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:19.287617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6671f1-9e6c-4179-b150-7d677db9b164 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:19.287906 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6671f1-9e6c-4179-b150-7d677db9b164 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:19.288187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6671f1-9e6c-4179-b150-7d677db9b164 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:19.288947 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6671f1-9e6c-4179-b150-7d677db9b164 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:19.289261 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6671f1-9e6c-4179-b150-7d677db9b164 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:19.290643 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6671f1-9e6c-4179-b150-7d677db9b164 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:19.303541 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 100/196] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:19.303541 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6671f1-9e6c-4179-b150-7d677db9b164 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:19.315192 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:19.342786 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 97/197] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:03:19.342786 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024ce6b-21be-48ce-90f5-ca4d967e5d86 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:19.363230 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:19.402403 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6671f1-9e6c-4179-b150-7d677db9b164 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:19.435740 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024ce6b-21be-48ce-90f5-ca4d967e5d86 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:19.437708 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277896c9-ea4f-40ba-97da-c0cbe00f3743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:19.438382 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277896c9-ea4f-40ba-97da-c0cbe00f3743 None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:19.438664 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277896c9-ea4f-40ba-97da-c0cbe00f3743 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:19.438967 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277896c9-ea4f-40ba-97da-c0cbe00f3743 None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:19.439681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277896c9-ea4f-40ba-97da-c0cbe00f3743 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:19.441123 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277896c9-ea4f-40ba-97da-c0cbe00f3743 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:19.469509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4d868-8a97-4b34-b162-6c220ad2a796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:19.470369 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4d868-8a97-4b34-b162-6c220ad2a796 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:19.470659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4d868-8a97-4b34-b162-6c220ad2a796 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:19.470958 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4d868-8a97-4b34-b162-6c220ad2a796 None admin] PATH_INFO: `/v3/users/ada63f4d813a4591a86492608c197e26` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:19.472147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd4d868-8a97-4b34-b162-6c220ad2a796 None admin] RBAC: Authorizing `identity:get_user(user_id=ada63f4d813a4591a86492608c197e26)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:19.473600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd4d868-8a97-4b34-b162-6c220ad2a796 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:19.475285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 98/198] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:19 2022] GET /identity/v3/users/ada63f4d813a4591a86492608c197e26 => generated 283 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:19.476041 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4d868-8a97-4b34-b162-6c220ad2a796 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:19.522225 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:19.572359 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4d868-8a97-4b34-b162-6c220ad2a796 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:19.578475 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 101/199] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jun 23 09:03:19 2022] POST /identity/v3/users => generated 297 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:03:19.579330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277896c9-ea4f-40ba-97da-c0cbe00f3743 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:19.610312 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fb55e-6a24-4252-aa1b-ea1d6cb7179b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:19.611250 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fb55e-6a24-4252-aa1b-ea1d6cb7179b None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:19.611536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fb55e-6a24-4252-aa1b-ea1d6cb7179b None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:19.611811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fb55e-6a24-4252-aa1b-ea1d6cb7179b None admin] PATH_INFO: `/v3/users/ca520b6a862c447987cd8f3b34dc58e7` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:19.613492 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fb55e-6a24-4252-aa1b-ea1d6cb7179b None admin] RBAC: Authorizing `identity:get_user(user_id=ca520b6a862c447987cd8f3b34dc58e7)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:19.615055 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fb55e-6a24-4252-aa1b-ea1d6cb7179b None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:19.617194 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 99/200] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:19 2022] GET /identity/v3/users/ca520b6a862c447987cd8f3b34dc58e7 => generated 289 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:19.618042 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fb55e-6a24-4252-aa1b-ea1d6cb7179b None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:19.693443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277896c9-ea4f-40ba-97da-c0cbe00f3743 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:19.712016 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:19.733785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fb55e-6a24-4252-aa1b-ea1d6cb7179b None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:19.734076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145e17d-adab-4a14-916a-04bcb3a93cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:19.735146 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145e17d-adab-4a14-916a-04bcb3a93cce None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:19.735772 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145e17d-adab-4a14-916a-04bcb3a93cce None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:19.735772 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145e17d-adab-4a14-916a-04bcb3a93cce None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:19.737387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145e17d-adab-4a14-916a-04bcb3a93cce None admin] RBAC: Authorizing `identity:get_project(project_id=c0336e9e466e4bd4a28254482978f7b2)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:19.738410 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145e17d-adab-4a14-916a-04bcb3a93cce None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:19.740195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 102/201] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:19 2022] GET /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:19.740969 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145e17d-adab-4a14-916a-04bcb3a93cce None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:19.750182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:19.775530 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25049f-9eca-4fa8-be40-0e4727a716b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:19.776160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25049f-9eca-4fa8-be40-0e4727a716b9 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:19.777370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25049f-9eca-4fa8-be40-0e4727a716b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:19.777370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25049f-9eca-4fa8-be40-0e4727a716b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:19.861918 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145e17d-adab-4a14-916a-04bcb3a93cce None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:19.865726 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25049f-9eca-4fa8-be40-0e4727a716b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:19.867284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25049f-9eca-4fa8-be40-0e4727a716b9 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:19.906889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6dd81-1f19-457d-9f8d-c897880b467d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:19.907585 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6dd81-1f19-457d-9f8d-c897880b467d None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:19.907871 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6dd81-1f19-457d-9f8d-c897880b467d None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:19.908150 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6dd81-1f19-457d-9f8d-c897880b467d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:19.972695 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:19.993006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 100/202] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:03:19.993343 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6dd81-1f19-457d-9f8d-c897880b467d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:19.994553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25049f-9eca-4fa8-be40-0e4727a716b9 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:19.999654 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6dd81-1f19-457d-9f8d-c897880b467d None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:20.015014 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:20.107626 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 103/203] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:03:20.108341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6dd81-1f19-457d-9f8d-c897880b467d None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:20.120784 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25049f-9eca-4fa8-be40-0e4727a716b9 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:20.162934 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e42e1f-66f6-477f-b938-cda82300f0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:20.162934 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e42e1f-66f6-477f-b938-cda82300f0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:20.162934 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e42e1f-66f6-477f-b938-cda82300f0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:20.162934 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e42e1f-66f6-477f-b938-cda82300f0a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:20.163332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e42e1f-66f6-477f-b938-cda82300f0a1 None admin] RBAC: Adding query filter params (role.id=9cd080b50c6f4ee8a54bf9ef1655bd83, scope.project.id=c0336e9e466e4bd4a28254482978f7b2, user.id=ada63f4d813a4591a86492608c197e26) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:20.163332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e42e1f-66f6-477f-b938-cda82300f0a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:20.163332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e42e1f-66f6-477f-b938-cda82300f0a1 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:20.170172 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 101/204] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:03:19 2022] GET /identity/v3/role_assignments?user.id=ada63f4d813a4591a86492608c197e26&scope.project.id=c0336e9e466e4bd4a28254482978f7b2&role.id=9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:20.170172 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e42e1f-66f6-477f-b938-cda82300f0a1 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:20.232045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:20.260824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:20.267647 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6dd81-1f19-457d-9f8d-c897880b467d None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:20.317205 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8675dd6-7b31-4973-8b6e-08fc6058c5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:20.318102 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8675dd6-7b31-4973-8b6e-08fc6058c5bd None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:20.318394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8675dd6-7b31-4973-8b6e-08fc6058c5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:20.318677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8675dd6-7b31-4973-8b6e-08fc6058c5bd None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:20.320210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8675dd6-7b31-4973-8b6e-08fc6058c5bd None admin] RBAC: Authorizing `identity:get_project(project_id=c0336e9e466e4bd4a28254482978f7b2)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:20.321720 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8675dd6-7b31-4973-8b6e-08fc6058c5bd None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:20.323907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 104/205] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:20 2022] GET /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:20.327455 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e42e1f-66f6-477f-b938-cda82300f0a1 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:20.334255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8675dd6-7b31-4973-8b6e-08fc6058c5bd None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:20.373222 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee205d8-32cc-45e4-b67e-a8a9155326c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:20.374498 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee205d8-32cc-45e4-b67e-a8a9155326c7 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:20.375026 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee205d8-32cc-45e4-b67e-a8a9155326c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:20.375523 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee205d8-32cc-45e4-b67e-a8a9155326c7 None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:20.376969 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee205d8-32cc-45e4-b67e-a8a9155326c7 None admin] RBAC: Authorizing `identity:get_project(project_id=79d2a60c71124efc90e0d0b445517d55)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:20.378633 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee205d8-32cc-45e4-b67e-a8a9155326c7 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:20.381092 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 102/206] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:20 2022] GET /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:20.386520 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee205d8-32cc-45e4-b67e-a8a9155326c7 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:20.463270 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8675dd6-7b31-4973-8b6e-08fc6058c5bd None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:20.500037 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f9af4-8434-482d-98b1-32432fa54837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:20.501070 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f9af4-8434-482d-98b1-32432fa54837 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:20.501560 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f9af4-8434-482d-98b1-32432fa54837 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:20.502033 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f9af4-8434-482d-98b1-32432fa54837 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:20.503147 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f9af4-8434-482d-98b1-32432fa54837 None admin] RBAC: Adding query filter params (role.id=478b327fd22845cc98a2fafe821b1e85, scope.project.id=c0336e9e466e4bd4a28254482978f7b2, user.id=ada63f4d813a4591a86492608c197e26) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:20.503670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f9af4-8434-482d-98b1-32432fa54837 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:20.505329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f9af4-8434-482d-98b1-32432fa54837 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:20.520630 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 105/207] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:03:20 2022] GET /identity/v3/role_assignments?user.id=ada63f4d813a4591a86492608c197e26&scope.project.id=c0336e9e466e4bd4a28254482978f7b2&role.id=478b327fd22845cc98a2fafe821b1e85 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:20.545626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee205d8-32cc-45e4-b67e-a8a9155326c7 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:20.583093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373e386-abf2-4bd0-88b7-40da93607c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:20.583969 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373e386-abf2-4bd0-88b7-40da93607c9e None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:20.584322 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373e386-abf2-4bd0-88b7-40da93607c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:20.584667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373e386-abf2-4bd0-88b7-40da93607c9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:20.585595 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373e386-abf2-4bd0-88b7-40da93607c9e None admin] RBAC: Adding query filter params (role.id=a6f4585744cf42e6b971891968972799, scope.project.id=79d2a60c71124efc90e0d0b445517d55, user.id=ca520b6a862c447987cd8f3b34dc58e7) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:20.585977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373e386-abf2-4bd0-88b7-40da93607c9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:20.587590 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373e386-abf2-4bd0-88b7-40da93607c9e None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:20.594361 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 103/208] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:03:20 2022] GET /identity/v3/role_assignments?user.id=ca520b6a862c447987cd8f3b34dc58e7&scope.project.id=79d2a60c71124efc90e0d0b445517d55&role.id=a6f4585744cf42e6b971891968972799 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:21.921415 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e72718-fd07-4f73-a3c7-2cae86102935 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:21.921963 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e72718-fd07-4f73-a3c7-2cae86102935 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:21.921963 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e72718-fd07-4f73-a3c7-2cae86102935 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:21.923399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 106/209] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:21.928662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a52b3-0b0c-4ae5-ab88-106581ceab4b None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:21.929450 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a52b3-0b0c-4ae5-ab88-106581ceab4b None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:21.929943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a52b3-0b0c-4ae5-ab88-106581ceab4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:21.946983 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-738a52b3-0b0c-4ae5-ab88-106581ceab4b None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:21.948464 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-738a52b3-0b0c-4ae5-ab88-106581ceab4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:21.951199 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:22.016970 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 104/210] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:03:22.040027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e72718-fd07-4f73-a3c7-2cae86102935 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:22.051311 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d37002-8567-42b2-a405-269d9e88aa07 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:22.051598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d37002-8567-42b2-a405-269d9e88aa07 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:22.051872 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d37002-8567-42b2-a405-269d9e88aa07 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:22.053564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 105/211] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:22.059242 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5794dbf-8ae8-4e89-a55d-0b5b2ea276f7 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:22.059544 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5794dbf-8ae8-4e89-a55d-0b5b2ea276f7 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:22.059826 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5794dbf-8ae8-4e89-a55d-0b5b2ea276f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:22.062234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:22.077071 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-d5794dbf-8ae8-4e89-a55d-0b5b2ea276f7 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:22.078268 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d5794dbf-8ae8-4e89-a55d-0b5b2ea276f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:22.155278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 106/212] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:03:22.178591 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e72718-fd07-4f73-a3c7-2cae86102935 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:22.180146 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5794dbf-8ae8-4e89-a55d-0b5b2ea276f7 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:22.222798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b91c4c-e0bd-46fd-8eb0-0218b7ac2de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:22.224020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b91c4c-e0bd-46fd-8eb0-0218b7ac2de4 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:22.224525 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b91c4c-e0bd-46fd-8eb0-0218b7ac2de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:22.224999 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b91c4c-e0bd-46fd-8eb0-0218b7ac2de4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:22.238450 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b91c4c-e0bd-46fd-8eb0-0218b7ac2de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:22.249340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-41b91c4c-e0bd-46fd-8eb0-0218b7ac2de4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:22.249340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:22.249340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:22.249340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:22.249340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:22.249340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:22.249340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:22.249340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:22.249340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:22.249340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:22.249340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:22.249340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:22.249340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:22.249340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:22.249340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:22.249340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:22.250293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:22.250293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:22.250293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:22.250293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:22.250293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:22.250293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:22.250293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:22.250293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:22.250293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:22.250293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:22.250293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:22.250293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:22.250293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:22.250293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:22.250293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:22.250293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:22.251179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:22.251179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:22.251179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:22.251179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:22.251179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:22.251179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:22.251179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:22.251179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:22.251179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:22.251179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:22.251179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:22.251179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:22.254128 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 107/213] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:22.259712 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b91c4c-e0bd-46fd-8eb0-0218b7ac2de4 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:22.260159 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:22.295609 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5794dbf-8ae8-4e89-a55d-0b5b2ea276f7 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:22.331519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0388a-3016-4ef0-98ed-0f7b7f42284f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:22.332923 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0388a-3016-4ef0-98ed-0f7b7f42284f None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:22.333207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0388a-3016-4ef0-98ed-0f7b7f42284f None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:22.333582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0388a-3016-4ef0-98ed-0f7b7f42284f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:22.335083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec0388a-3016-4ef0-98ed-0f7b7f42284f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:22.336639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec0388a-3016-4ef0-98ed-0f7b7f42284f None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:22.338589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 107/214] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:03:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:22.344598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0388a-3016-4ef0-98ed-0f7b7f42284f None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:22.357877 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:22.381627 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b91c4c-e0bd-46fd-8eb0-0218b7ac2de4 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:22.416195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994062b-e0b3-4333-975b-cd023227638a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:22.417124 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994062b-e0b3-4333-975b-cd023227638a None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:22.417569 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994062b-e0b3-4333-975b-cd023227638a None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:22.417848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994062b-e0b3-4333-975b-cd023227638a None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:22.418628 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994062b-e0b3-4333-975b-cd023227638a None admin] RBAC: Adding query filter params (name=member) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:22.419050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994062b-e0b3-4333-975b-cd023227638a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:22.420330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994062b-e0b3-4333-975b-cd023227638a None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:22.433816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 108/215] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:22.433816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dfe12-0dfc-49b6-b617-ac9b0a181671 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:22.433816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dfe12-0dfc-49b6-b617-ac9b0a181671 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:22.433816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dfe12-0dfc-49b6-b617-ac9b0a181671 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:22.433816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 109/216] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:22.436186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279dfe12-0dfc-49b6-b617-ac9b0a181671 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:22.461214 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0388a-3016-4ef0-98ed-0f7b7f42284f None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:22.503872 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8f8bd-5ab6-46a1-b61d-2c98193eb494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:22.504720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8f8bd-5ab6-46a1-b61d-2c98193eb494 None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:22.505014 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8f8bd-5ab6-46a1-b61d-2c98193eb494 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:22.505552 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8f8bd-5ab6-46a1-b61d-2c98193eb494 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:22.506007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8f8bd-5ab6-46a1-b61d-2c98193eb494 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:22.507493 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8f8bd-5ab6-46a1-b61d-2c98193eb494 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:22.515627 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:22.556239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279dfe12-0dfc-49b6-b617-ac9b0a181671 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:22.591132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60497812-b83d-4d77-ac3b-34e8e47b3435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:22.592037 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60497812-b83d-4d77-ac3b-34e8e47b3435 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:22.592324 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60497812-b83d-4d77-ac3b-34e8e47b3435 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:22.592602 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60497812-b83d-4d77-ac3b-34e8e47b3435 None admin] PATH_INFO: `/v3/users/942cda9cd07c437db4064f34a91e2c70` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:22.600578 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:22.606838 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60497812-b83d-4d77-ac3b-34e8e47b3435 None admin] RBAC: Authorizing `identity:get_user(user_id=942cda9cd07c437db4064f34a91e2c70)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:22.608340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60497812-b83d-4d77-ac3b-34e8e47b3435 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:22.610022 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 110/217] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:22 2022] GET /identity/v3/users/942cda9cd07c437db4064f34a91e2c70 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:22.611210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d15f91-7dd6-4ebb-b3d0-e49a625e913b None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:22.611501 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d15f91-7dd6-4ebb-b3d0-e49a625e913b None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:22.611774 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d15f91-7dd6-4ebb-b3d0-e49a625e913b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:22.621805 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 108/218] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jun 23 09:03:22 2022] POST /identity/v3/users => generated 297 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:03:22.622857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8f8bd-5ab6-46a1-b61d-2c98193eb494 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:22.636614 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-e4d15f91-7dd6-4ebb-b3d0-e49a625e913b None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:22.639007 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e4d15f91-7dd6-4ebb-b3d0-e49a625e913b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:22.722608 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 111/219] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:22.724013 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20053aaf-9b11-45ec-99b0-2342b431713d None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:22.724304 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20053aaf-9b11-45ec-99b0-2342b431713d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:22.724584 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20053aaf-9b11-45ec-99b0-2342b431713d None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:22.726066 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 112/220] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:22.732088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1585aca-ff67-458b-a216-722bd788749c None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:22.732376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1585aca-ff67-458b-a216-722bd788749c None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:22.732661 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1585aca-ff67-458b-a216-722bd788749c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:22.751585 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-f1585aca-ff67-458b-a216-722bd788749c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:22.752564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8f8bd-5ab6-46a1-b61d-2c98193eb494 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:22.753276 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-f1585aca-ff67-458b-a216-722bd788749c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:22.792661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317ae20-3c0b-4d44-b28b-87bc90230334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:22.793492 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317ae20-3c0b-4d44-b28b-87bc90230334 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:22.794409 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317ae20-3c0b-4d44-b28b-87bc90230334 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:22.794825 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317ae20-3c0b-4d44-b28b-87bc90230334 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:22.831995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 113/221] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:03:22.833268 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1585aca-ff67-458b-a216-722bd788749c None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:22.871456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:22.883619 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317ae20-3c0b-4d44-b28b-87bc90230334 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:22.885126 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317ae20-3c0b-4d44-b28b-87bc90230334 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:22.904247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:22.962262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1585aca-ff67-458b-a216-722bd788749c None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:23.002698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a2317-b169-491d-a81e-7c044cdb910b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:23.003454 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a2317-b169-491d-a81e-7c044cdb910b None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:23.003742 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a2317-b169-491d-a81e-7c044cdb910b None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:23.004020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a2317-b169-491d-a81e-7c044cdb910b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:23.018765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a2317-b169-491d-a81e-7c044cdb910b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:23.019936 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 109/222] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:03:23.021332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317ae20-3c0b-4d44-b28b-87bc90230334 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:23.028883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-5a7a2317-b169-491d-a81e-7c044cdb910b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:03:23.028883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:23.028883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:23.028883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:23.028883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:23.028883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:23.028883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:23.028883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:23.028883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:23.028883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:23.028883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:23.028883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:23.028883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:23.028883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:23.028883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:23.028883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:23.030054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:23.030054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:23.030054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:23.030054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:23.030054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:23.030054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:23.030054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:23.030054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:23.030054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:23.030054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:23.030054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:23.030054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:23.030054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:23.030054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:23.030054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:23.030054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:23.031092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:23.031092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:23.031092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:23.031092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:23.031092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:23.031092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:23.031092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:23.031092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:23.031092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:23.031092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:23.031092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:03:23.031092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:23.032511 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 114/223] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:03:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:23.033270 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a2317-b169-491d-a81e-7c044cdb910b None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:23.125151 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:23.134306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317ae20-3c0b-4d44-b28b-87bc90230334 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:23.152602 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a2317-b169-491d-a81e-7c044cdb910b None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:23.171518 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:23.174679 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b2f64-0e3a-43c0-a31c-24ec6ddaf97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:23.175531 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b2f64-0e3a-43c0-a31c-24ec6ddaf97a None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:23.175822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b2f64-0e3a-43c0-a31c-24ec6ddaf97a None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:23.176102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b2f64-0e3a-43c0-a31c-24ec6ddaf97a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:23.188093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b2f64-0e3a-43c0-a31c-24ec6ddaf97a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:23.193935 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9be01-8065-468c-8693-06af3b940828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:23.195103 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9be01-8065-468c-8693-06af3b940828 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:23.195599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9be01-8065-468c-8693-06af3b940828 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:23.196105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9be01-8065-468c-8693-06af3b940828 None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:23.196714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-d33b2f64-0e3a-43c0-a31c-24ec6ddaf97a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:23.196714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:23.196714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:23.196714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:23.196714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:23.196714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:23.196714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:23.196714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:23.196714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:23.196714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:23.196714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:23.196714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:23.196714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:23.196714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:23.196714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:23.196714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:23.197640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:23.197640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:23.197640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:23.197640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:23.197640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:23.197640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:23.197640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:23.197640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:23.197640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:23.197640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:23.197640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:23.197640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:23.197640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:23.197640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:23.197640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:23.197640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:23.198494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:23.198494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:23.198494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:23.198494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:23.198494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:23.198494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:23.198494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:23.198494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:23.198494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:23.198494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:23.198494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:23.198494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:23.198494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 110/224] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:03:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:23.199324 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b2f64-0e3a-43c0-a31c-24ec6ddaf97a None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:23.206076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9be01-8065-468c-8693-06af3b940828 None admin] RBAC: Authorizing `identity:get_project(project_id=c0336e9e466e4bd4a28254482978f7b2)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:23.209153 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9be01-8065-468c-8693-06af3b940828 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:23.211127 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 115/225] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:23 2022] GET /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:23.215887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9be01-8065-468c-8693-06af3b940828 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:23.322572 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9be01-8065-468c-8693-06af3b940828 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:23.323845 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b2f64-0e3a-43c0-a31c-24ec6ddaf97a None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:23.361482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f718e7-1b72-4793-9559-cef1ef9f05da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:23.361796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee87b2-94e8-4a90-b736-6aa8dd626081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:23.362335 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f718e7-1b72-4793-9559-cef1ef9f05da None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:23.362721 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f718e7-1b72-4793-9559-cef1ef9f05da None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:23.363095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f718e7-1b72-4793-9559-cef1ef9f05da None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:23.363369 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee87b2-94e8-4a90-b736-6aa8dd626081 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:23.363822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee87b2-94e8-4a90-b736-6aa8dd626081 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:23.364031 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f718e7-1b72-4793-9559-cef1ef9f05da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:23.364416 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f718e7-1b72-4793-9559-cef1ef9f05da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:23.364634 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee87b2-94e8-4a90-b736-6aa8dd626081 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:23.365561 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee87b2-94e8-4a90-b736-6aa8dd626081 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:23.366017 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f718e7-1b72-4793-9559-cef1ef9f05da None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:23.366224 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee87b2-94e8-4a90-b736-6aa8dd626081 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:23.367999 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee87b2-94e8-4a90-b736-6aa8dd626081 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:23.380484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 116/226] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:03:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:23.380484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f718e7-1b72-4793-9559-cef1ef9f05da None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:23.382254 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 111/227] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:23.383294 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee87b2-94e8-4a90-b736-6aa8dd626081 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:23.402393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:23.438141 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:23.496517 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f718e7-1b72-4793-9559-cef1ef9f05da None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:23.513013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee87b2-94e8-4a90-b736-6aa8dd626081 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:23.535810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53e6f9-d4a6-499e-b167-11058b28df91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:23.536901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53e6f9-d4a6-499e-b167-11058b28df91 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:23.537401 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53e6f9-d4a6-499e-b167-11058b28df91 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:23.537971 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53e6f9-d4a6-499e-b167-11058b28df91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:23.539102 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53e6f9-d4a6-499e-b167-11058b28df91 None admin] RBAC: Adding query filter params (role.id=53a3d65343394e7caafcc33f12cf138b, scope.project.id=c0336e9e466e4bd4a28254482978f7b2, user.id=942cda9cd07c437db4064f34a91e2c70) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:23.539615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53e6f9-d4a6-499e-b167-11058b28df91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:23.541466 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53e6f9-d4a6-499e-b167-11058b28df91 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:23.554704 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 117/228] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:03:23 2022] GET /identity/v3/role_assignments?user.id=942cda9cd07c437db4064f34a91e2c70&scope.project.id=c0336e9e466e4bd4a28254482978f7b2&role.id=53a3d65343394e7caafcc33f12cf138b => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:23.554704 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53e6f9-d4a6-499e-b167-11058b28df91 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:23.560414 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48400f9-73e5-431a-9175-d212c5232923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:23.561334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48400f9-73e5-431a-9175-d212c5232923 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:23.561622 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48400f9-73e5-431a-9175-d212c5232923 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:23.561904 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48400f9-73e5-431a-9175-d212c5232923 None admin] PATH_INFO: `/v3/users/ada63f4d813a4591a86492608c197e26` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:23.569260 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48400f9-73e5-431a-9175-d212c5232923 None admin] RBAC: Authorizing `identity:get_user(user_id=ada63f4d813a4591a86492608c197e26)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:23.570775 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48400f9-73e5-431a-9175-d212c5232923 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:23.576186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 112/229] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:23 2022] GET /identity/v3/users/ada63f4d813a4591a86492608c197e26 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:23.579000 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48400f9-73e5-431a-9175-d212c5232923 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:23.737056 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:23.773720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48400f9-73e5-431a-9175-d212c5232923 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:23.776154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53e6f9-d4a6-499e-b167-11058b28df91 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:23.787882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:23.822598 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cfe4b-fe84-4c94-9390-1e3221d2a95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:23.824046 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cfe4b-fe84-4c94-9390-1e3221d2a95f None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:23.824559 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cfe4b-fe84-4c94-9390-1e3221d2a95f None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:23.825035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cfe4b-fe84-4c94-9390-1e3221d2a95f None admin] PATH_INFO: `/v3/users/ada63f4d813a4591a86492608c197e26` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:23.826536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11cfe4b-fe84-4c94-9390-1e3221d2a95f None admin] RBAC: Authorizing `identity:get_user(user_id=ada63f4d813a4591a86492608c197e26)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:23.828370 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11cfe4b-fe84-4c94-9390-1e3221d2a95f None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:23.828952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be99d66-cdcd-4cad-92de-ec257aa6309b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:23.829920 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be99d66-cdcd-4cad-92de-ec257aa6309b None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:23.830305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be99d66-cdcd-4cad-92de-ec257aa6309b None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:23.830754 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be99d66-cdcd-4cad-92de-ec257aa6309b None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:23.831147 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 118/230] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:23 2022] GET /identity/v3/users/ada63f4d813a4591a86492608c197e26 => generated 283 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:23.835514 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be99d66-cdcd-4cad-92de-ec257aa6309b None admin] RBAC: Authorizing `identity:get_project(project_id=c0336e9e466e4bd4a28254482978f7b2)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:23.835514 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be99d66-cdcd-4cad-92de-ec257aa6309b None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:23.836090 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 113/231] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:23 2022] GET /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:23.836786 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cfe4b-fe84-4c94-9390-1e3221d2a95f None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:23.847909 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be99d66-cdcd-4cad-92de-ec257aa6309b None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:23.984344 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cfe4b-fe84-4c94-9390-1e3221d2a95f None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:24.040738 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be99d66-cdcd-4cad-92de-ec257aa6309b None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:24.052225 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd9b61-2311-4c1f-936c-14f98e5d76e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:24.053426 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd9b61-2311-4c1f-936c-14f98e5d76e4 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:24.053919 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd9b61-2311-4c1f-936c-14f98e5d76e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:24.054396 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd9b61-2311-4c1f-936c-14f98e5d76e4 None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:24.057758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd9b61-2311-4c1f-936c-14f98e5d76e4 None admin] RBAC: Authorizing `identity:get_project(project_id=c0336e9e466e4bd4a28254482978f7b2)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:24.060089 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd9b61-2311-4c1f-936c-14f98e5d76e4 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:24.064169 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 119/232] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:23 2022] GET /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:24.069904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd9b61-2311-4c1f-936c-14f98e5d76e4 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:24.106678 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a8a21-91d4-4344-b00d-009ee8b6ca7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:24.107941 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a8a21-91d4-4344-b00d-009ee8b6ca7a None admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:24.109345 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a8a21-91d4-4344-b00d-009ee8b6ca7a None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:24.109848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a8a21-91d4-4344-b00d-009ee8b6ca7a None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2/users/ada63f4d813a4591a86492608c197e26/roles/478b327fd22845cc98a2fafe821b1e85` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:24.113932 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a8a21-91d4-4344-b00d-009ee8b6ca7a None admin] RBAC: Authorizing `identity:create_grant(project_id=c0336e9e466e4bd4a28254482978f7b2, user_id=ada63f4d813a4591a86492608c197e26, role_id=478b327fd22845cc98a2fafe821b1e85)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:24.116123 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a8a21-91d4-4344-b00d-009ee8b6ca7a None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:24.157003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:24.197982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd9b61-2311-4c1f-936c-14f98e5d76e4 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:24.231865 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4897701-d064-4a3e-8686-dc027e017f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:24.233077 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4897701-d064-4a3e-8686-dc027e017f48 None admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:24.233616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4897701-d064-4a3e-8686-dc027e017f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:24.236041 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4897701-d064-4a3e-8686-dc027e017f48 None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2/users/ada63f4d813a4591a86492608c197e26/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:24.241117 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 114/233] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:03:23 2022] PUT /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2/users/ada63f4d813a4591a86492608c197e26/roles/478b327fd22845cc98a2fafe821b1e85 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:03:24.243165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4897701-d064-4a3e-8686-dc027e017f48 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0336e9e466e4bd4a28254482978f7b2, user_id=ada63f4d813a4591a86492608c197e26, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:24.246431 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4897701-d064-4a3e-8686-dc027e017f48 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:24.342606 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 120/234] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:03:24 2022] PUT /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2/users/ada63f4d813a4591a86492608c197e26/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:03:25.072516 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00349791-fb5c-45c3-95d7-0bb5713c8cd8 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:25.073034 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00349791-fb5c-45c3-95d7-0bb5713c8cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:25.073112 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00349791-fb5c-45c3-95d7-0bb5713c8cd8 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:25.075367 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 115/235] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:25.092667 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b27b4-c1ad-45c6-aa76-720d8cc528f9 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:25.093159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b27b4-c1ad-45c6-aa76-720d8cc528f9 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:25.093237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b27b4-c1ad-45c6-aa76-720d8cc528f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:25.098358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c51a28-e878-4703-a22a-cdb4d1c38fa5 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:25.098648 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c51a28-e878-4703-a22a-cdb4d1c38fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:25.098944 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c51a28-e878-4703-a22a-cdb4d1c38fa5 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:25.100671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 116/236] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:25.105644 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b36cf-384d-45b5-82cf-8784c83563ef None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:25.105644 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b36cf-384d-45b5-82cf-8784c83563ef None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:25.105848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b36cf-384d-45b5-82cf-8784c83563ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:25.123880 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-ac3b27b4-c1ad-45c6-aa76-720d8cc528f9 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:25.125083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-ac3b27b4-c1ad-45c6-aa76-720d8cc528f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:25.137159 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-8b1b36cf-384d-45b5-82cf-8784c83563ef None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:25.138362 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-8b1b36cf-384d-45b5-82cf-8784c83563ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:25.143099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:25.206818 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 121/237] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:25.231134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b27b4-c1ad-45c6-aa76-720d8cc528f9 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:25.266716 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:25.276564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 117/238] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:25.326075 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b36cf-384d-45b5-82cf-8784c83563ef None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:25.336137 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b27b4-c1ad-45c6-aa76-720d8cc528f9 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:25.386224 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479b70d-3ca4-4ab6-aac9-8b3584c65699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:25.387561 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479b70d-3ca4-4ab6-aac9-8b3584c65699 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:25.390080 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479b70d-3ca4-4ab6-aac9-8b3584c65699 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:25.390550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479b70d-3ca4-4ab6-aac9-8b3584c65699 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:25.410094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479b70d-3ca4-4ab6-aac9-8b3584c65699 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:25.418749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-1479b70d-3ca4-4ab6-aac9-8b3584c65699 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 09:03:25.418749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:25.418749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:25.418749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:25.418749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:25.418749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:25.418749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:25.418749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:25.418749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:25.418749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:25.418749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:25.418749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:25.418749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:25.418749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:25.418749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:25.418749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:25.419744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:25.419744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:25.419744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:25.419744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:25.419744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:25.419744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:25.419744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:25.419744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:25.419744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:25.419744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:25.419744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:25.419744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:25.419744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:25.419744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:25.419744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:25.419744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:25.420606 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:25.420606 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:25.420606 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:25.420606 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:25.420606 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:25.420606 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:25.420606 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:25.420606 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:25.420606 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:25.420606 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:25.420606 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 09:03:25.420606 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:25.422898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 122/239] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:03:25.428710 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479b70d-3ca4-4ab6-aac9-8b3584c65699 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:25.539058 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479b70d-3ca4-4ab6-aac9-8b3584c65699 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:25.541038 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:25.579545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563df562-76ec-460f-9ab1-8b8041298b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:25.580736 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563df562-76ec-460f-9ab1-8b8041298b5a None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:25.581287 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563df562-76ec-460f-9ab1-8b8041298b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:25.581719 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563df562-76ec-460f-9ab1-8b8041298b5a None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:25.582948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563df562-76ec-460f-9ab1-8b8041298b5a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:25.583513 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563df562-76ec-460f-9ab1-8b8041298b5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:25.585011 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b36cf-384d-45b5-82cf-8784c83563ef None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:25.585266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563df562-76ec-460f-9ab1-8b8041298b5a None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:25.595585 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 123/240] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:25.601440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563df562-76ec-460f-9ab1-8b8041298b5a None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:25.669271 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1ae81-a5af-4381-8df2-953e6f8dec59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:25.670367 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1ae81-a5af-4381-8df2-953e6f8dec59 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:25.670803 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1ae81-a5af-4381-8df2-953e6f8dec59 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:25.671260 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1ae81-a5af-4381-8df2-953e6f8dec59 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:25.672601 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1ae81-a5af-4381-8df2-953e6f8dec59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:25.675715 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1ae81-a5af-4381-8df2-953e6f8dec59 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:25.679181 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 118/241] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:03:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:25.684712 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1ae81-a5af-4381-8df2-953e6f8dec59 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:25.705191 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:25.787284 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563df562-76ec-460f-9ab1-8b8041298b5a None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:25.847297 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1ae81-a5af-4381-8df2-953e6f8dec59 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:25.879226 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ce300-431f-4b00-b616-3b83ef0b77ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:25.880117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ce300-431f-4b00-b616-3b83ef0b77ab None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:25.880410 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ce300-431f-4b00-b616-3b83ef0b77ab None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:25.880685 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ce300-431f-4b00-b616-3b83ef0b77ab None admin] PATH_INFO: `/v3/users/788b5a145ce24c9999b6880a95294f99` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:25.886541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca9ac8-c5b0-467f-b5b2-171ea9bdcd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:25.887280 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca9ac8-c5b0-467f-b5b2-171ea9bdcd28 None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:25.887570 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca9ac8-c5b0-467f-b5b2-171ea9bdcd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:25.887849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca9ac8-c5b0-467f-b5b2-171ea9bdcd28 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:25.888568 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca9ac8-c5b0-467f-b5b2-171ea9bdcd28 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:25.890731 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca9ac8-c5b0-467f-b5b2-171ea9bdcd28 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:25.894701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ce300-431f-4b00-b616-3b83ef0b77ab None admin] RBAC: Authorizing `identity:get_user(user_id=788b5a145ce24c9999b6880a95294f99)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:25.896291 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ce300-431f-4b00-b616-3b83ef0b77ab None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:25.898324 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 124/242] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:25 2022] GET /identity/v3/users/788b5a145ce24c9999b6880a95294f99 => generated 297 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:25.908053 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ce300-431f-4b00-b616-3b83ef0b77ab None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:25.933693 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:25.988003 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 119/243] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jun 23 09:03:25 2022] POST /identity/v3/users => generated 293 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:03:26.003006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab0215-a671-4d24-bea8-380a92561f78 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:26.003304 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab0215-a671-4d24-bea8-380a92561f78 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:26.003590 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab0215-a671-4d24-bea8-380a92561f78 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:26.005333 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 120/244] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:03:26.016759 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbf3d1-c777-48d2-af81-83ae9045ac10 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:26.017138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbf3d1-c777-48d2-af81-83ae9045ac10 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:26.017329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbf3d1-c777-48d2-af81-83ae9045ac10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:26.037101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ce300-431f-4b00-b616-3b83ef0b77ab None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:26.039242 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-82bbf3d1-c777-48d2-af81-83ae9045ac10 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:26.041468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-82bbf3d1-c777-48d2-af81-83ae9045ac10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:26.120049 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7e405-578f-4e8e-bf97-7b5d1af56bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:26.120712 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7e405-578f-4e8e-bf97-7b5d1af56bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:26.120928 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7e405-578f-4e8e-bf97-7b5d1af56bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:26.121212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7e405-578f-4e8e-bf97-7b5d1af56bb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:26.138447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:26.154157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 121/245] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:03:26.192382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbf3d1-c777-48d2-af81-83ae9045ac10 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:26.230622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7e405-578f-4e8e-bf97-7b5d1af56bb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:26.232272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7e405-578f-4e8e-bf97-7b5d1af56bb7 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:26.250319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:26.334809 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbf3d1-c777-48d2-af81-83ae9045ac10 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:26.376330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 125/246] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 09:03:26.382298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb0731-4e15-45d8-b4d4-e39fd301b9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:26.382886 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7e405-578f-4e8e-bf97-7b5d1af56bb7 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:26.383145 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb0731-4e15-45d8-b4d4-e39fd301b9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:26.383430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb0731-4e15-45d8-b4d4-e39fd301b9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:26.383707 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb0731-4e15-45d8-b4d4-e39fd301b9e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:26.397660 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb0731-4e15-45d8-b4d4-e39fd301b9e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:26.405503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-cdbb0731-4e15-45d8-b4d4-e39fd301b9e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:26.405503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:26.405503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:26.405503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:26.405503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:26.405503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:26.405503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:26.405503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:26.405503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:26.405503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:26.405503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:26.405503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:26.405503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:26.405503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:26.405503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:26.405503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:26.406655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:26.406655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:26.406655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:26.406655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:26.406655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:26.406655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:26.406655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:26.406655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:26.406655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:26.406655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:26.406655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:26.406655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:26.406655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:26.406655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:26.406655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:26.406655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:26.407769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:26.407769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:26.407769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:26.407769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:26.407769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:26.407769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:26.407769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:26.407769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:26.407769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:26.407769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:26.407769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:26.407769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:26.407769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 122/247] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:03:26.413312 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb0731-4e15-45d8-b4d4-e39fd301b9e6 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:26.483656 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:26.504785 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7e405-578f-4e8e-bf97-7b5d1af56bb7 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:26.524224 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:26.534704 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb0731-4e15-45d8-b4d4-e39fd301b9e6 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:26.545715 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27955c-067c-4c10-a70f-dcf20026cda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:26.546872 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27955c-067c-4c10-a70f-dcf20026cda2 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:26.547356 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27955c-067c-4c10-a70f-dcf20026cda2 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:26.547785 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27955c-067c-4c10-a70f-dcf20026cda2 None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:26.549206 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b27955c-067c-4c10-a70f-dcf20026cda2 None admin] RBAC: Authorizing `identity:get_project(project_id=c0336e9e466e4bd4a28254482978f7b2)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:26.550837 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b27955c-067c-4c10-a70f-dcf20026cda2 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:26.554007 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 126/248] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:26 2022] GET /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:26.559940 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27955c-067c-4c10-a70f-dcf20026cda2 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:26.588612 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0cd22b-e466-4622-94bc-a3488dde77a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:26.589512 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0cd22b-e466-4622-94bc-a3488dde77a2 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:26.589911 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0cd22b-e466-4622-94bc-a3488dde77a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:26.590287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0cd22b-e466-4622-94bc-a3488dde77a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:26.591203 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0cd22b-e466-4622-94bc-a3488dde77a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:26.591617 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0cd22b-e466-4622-94bc-a3488dde77a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:26.593147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0cd22b-e466-4622-94bc-a3488dde77a2 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:26.609060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 123/249] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:26.614364 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0cd22b-e466-4622-94bc-a3488dde77a2 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:26.692866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27955c-067c-4c10-a70f-dcf20026cda2 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:26.727369 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0cd22b-e466-4622-94bc-a3488dde77a2 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:26.750160 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2645a-cfb8-4e1e-975e-33f2640f3de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:26.751100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2645a-cfb8-4e1e-975e-33f2640f3de1 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:26.751540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2645a-cfb8-4e1e-975e-33f2640f3de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:26.751995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2645a-cfb8-4e1e-975e-33f2640f3de1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:26.753209 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea2645a-cfb8-4e1e-975e-33f2640f3de1 None admin] RBAC: Adding query filter params (role.id=a6f4585744cf42e6b971891968972799, scope.project.id=c0336e9e466e4bd4a28254482978f7b2, user.id=788b5a145ce24c9999b6880a95294f99) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:26.753674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea2645a-cfb8-4e1e-975e-33f2640f3de1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:26.756180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea2645a-cfb8-4e1e-975e-33f2640f3de1 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:26.759822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:26.769344 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 127/250] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:03:26 2022] GET /identity/v3/role_assignments?user.id=788b5a145ce24c9999b6880a95294f99&scope.project.id=c0336e9e466e4bd4a28254482978f7b2&role.id=a6f4585744cf42e6b971891968972799 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:26.772069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5698d1-6604-42e5-8d5e-5b699076393f None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:26.774216 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5698d1-6604-42e5-8d5e-5b699076393f None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:26.775290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5698d1-6604-42e5-8d5e-5b699076393f None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:26.777638 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 128/251] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:26.784116 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea139c21-6417-44c1-81e3-57aedcd41fc2 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:26.784631 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea139c21-6417-44c1-81e3-57aedcd41fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:26.785084 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea139c21-6417-44c1-81e3-57aedcd41fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:26.805226 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275f0a1-4d3c-4473-a7b2-cf05c8b5ae85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:26.806383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275f0a1-4d3c-4473-a7b2-cf05c8b5ae85 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:26.806852 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275f0a1-4d3c-4473-a7b2-cf05c8b5ae85 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:26.807309 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275f0a1-4d3c-4473-a7b2-cf05c8b5ae85 None admin] PATH_INFO: `/v3/users/942cda9cd07c437db4064f34a91e2c70` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:26.808727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275f0a1-4d3c-4473-a7b2-cf05c8b5ae85 None admin] RBAC: Authorizing `identity:get_user(user_id=942cda9cd07c437db4064f34a91e2c70)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:26.810946 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275f0a1-4d3c-4473-a7b2-cf05c8b5ae85 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:26.815845 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 124/252] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:26 2022] GET /identity/v3/users/942cda9cd07c437db4064f34a91e2c70 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:26.819902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-ea139c21-6417-44c1-81e3-57aedcd41fc2 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:26.821290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275f0a1-4d3c-4473-a7b2-cf05c8b5ae85 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:26.823116 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-ea139c21-6417-44c1-81e3-57aedcd41fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:26.892575 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:26.921275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 129/253] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:26.958179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea139c21-6417-44c1-81e3-57aedcd41fc2 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:27.061034 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275f0a1-4d3c-4473-a7b2-cf05c8b5ae85 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:27.086812 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea139c21-6417-44c1-81e3-57aedcd41fc2 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:27.110319 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c318c75-bb70-41f6-a841-cfa2768d6c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:27.111751 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c318c75-bb70-41f6-a841-cfa2768d6c2b None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:27.112046 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c318c75-bb70-41f6-a841-cfa2768d6c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:27.112328 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c318c75-bb70-41f6-a841-cfa2768d6c2b None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:27.113580 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c318c75-bb70-41f6-a841-cfa2768d6c2b None admin] RBAC: Authorizing `identity:get_project(project_id=c0336e9e466e4bd4a28254482978f7b2)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:27.115578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c318c75-bb70-41f6-a841-cfa2768d6c2b None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:27.117484 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 125/254] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:26 2022] GET /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2 => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:27.118803 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c3a0a-7801-43da-8660-b71fe563a139 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:27.119130 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c3a0a-7801-43da-8660-b71fe563a139 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:27.119420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c3a0a-7801-43da-8660-b71fe563a139 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:27.120842 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 126/255] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:27.125863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c3a0a-7801-43da-8660-b71fe563a139 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:27.155225 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b5b4b-62c6-4897-bcb9-9019fb704013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:27.156059 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b5b4b-62c6-4897-bcb9-9019fb704013 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:27.156350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b5b4b-62c6-4897-bcb9-9019fb704013 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:27.156629 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b5b4b-62c6-4897-bcb9-9019fb704013 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:27.173199 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b5b4b-62c6-4897-bcb9-9019fb704013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:27.181699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-695b5b4b-62c6-4897-bcb9-9019fb704013 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:27.181699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:27.181699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:27.181699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:27.181699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:27.181699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:27.181699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:27.181699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:27.181699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:27.181699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:27.181699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:27.181699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:27.181699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:27.181699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:27.181699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:27.181699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:27.182651 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:27.182651 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:27.182651 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:27.182651 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:27.182651 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:27.182651 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:27.182651 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:27.182651 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:27.182651 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:27.182651 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:27.182651 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:27.182651 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:27.182651 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:27.182651 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:27.182651 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:27.182651 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:27.183546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:27.183546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:27.183546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:27.183546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:27.183546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:27.183546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:27.183546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:27.183546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:27.183546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:27.183546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:27.183546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:27.183546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:27.183546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 130/256] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:03:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:03:27.184816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c89c55-fbdf-4289-b996-bbf3aac25691 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:27.184816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c89c55-fbdf-4289-b996-bbf3aac25691 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:27.185176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c89c55-fbdf-4289-b996-bbf3aac25691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:27.456214 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-67c89c55-fbdf-4289-b996-bbf3aac25691 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:27.459917 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-67c89c55-fbdf-4289-b996-bbf3aac25691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:27.502463 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:27.519913 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:27.583074 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c3a0a-7801-43da-8660-b71fe563a139 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:27.599152 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 131/257] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:27.600773 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c89c55-fbdf-4289-b996-bbf3aac25691 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:27.664009 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374af3d-435f-4fa6-a4f3-8c519f9fec05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:27.665965 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374af3d-435f-4fa6-a4f3-8c519f9fec05 None admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:27.669748 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374af3d-435f-4fa6-a4f3-8c519f9fec05 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:27.670225 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374af3d-435f-4fa6-a4f3-8c519f9fec05 None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2/users/942cda9cd07c437db4064f34a91e2c70/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:27.673333 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374af3d-435f-4fa6-a4f3-8c519f9fec05 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0336e9e466e4bd4a28254482978f7b2, user_id=942cda9cd07c437db4064f34a91e2c70, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:27.680482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374af3d-435f-4fa6-a4f3-8c519f9fec05 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:27.749053 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c89c55-fbdf-4289-b996-bbf3aac25691 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:27.756968 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 127/258] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:03:27 2022] PUT /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2/users/942cda9cd07c437db4064f34a91e2c70/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:03:27.764300 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374af3d-435f-4fa6-a4f3-8c519f9fec05 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:27.800013 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1cf5c-7768-4ad8-a04a-32f577240609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:27.800875 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1cf5c-7768-4ad8-a04a-32f577240609 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:27.801170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1cf5c-7768-4ad8-a04a-32f577240609 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:27.801452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1cf5c-7768-4ad8-a04a-32f577240609 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:27.802506 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1cf5c-7768-4ad8-a04a-32f577240609 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:27.802686 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1cf5c-7768-4ad8-a04a-32f577240609 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:27.804159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1cf5c-7768-4ad8-a04a-32f577240609 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:27.819801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 132/259] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:03:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:27.830960 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1cf5c-7768-4ad8-a04a-32f577240609 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:27.882494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:27.955231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374af3d-435f-4fa6-a4f3-8c519f9fec05 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:27.978437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:28.022886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a65d3-c178-4565-aee0-65a114c5fd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:28.023641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a65d3-c178-4565-aee0-65a114c5fd9b None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:28.023915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a65d3-c178-4565-aee0-65a114c5fd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:28.024197 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a65d3-c178-4565-aee0-65a114c5fd9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:28.048661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a65d3-c178-4565-aee0-65a114c5fd9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:28.062994 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1cf5c-7768-4ad8-a04a-32f577240609 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:28.064022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-012a65d3-c178-4565-aee0-65a114c5fd9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:03:28.064022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:28.064022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:28.064022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:28.064022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:28.064022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:28.064022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:28.064022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:28.064022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:28.064022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:28.064022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:28.064022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:28.064022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:28.064022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:28.064022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:28.064022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:28.064973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:28.064973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:28.064973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:28.064973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:28.064973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:28.064973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:28.064973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:28.064973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:28.064973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:28.064973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:28.064973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:28.064973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:28.064973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:28.064973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:28.064973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:28.064973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:28.065835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:28.065835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:28.065835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:28.065835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:28.065835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:28.065835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:28.065835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:28.065835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:28.065835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:28.065835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:28.065835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:03:28.065835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:28.068223 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 128/260] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:03:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:28.074012 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a65d3-c178-4565-aee0-65a114c5fd9b None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:28.175650 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ff57c-700a-41c5-90fc-dfa861559116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:28.176553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ff57c-700a-41c5-90fc-dfa861559116 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:28.176848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ff57c-700a-41c5-90fc-dfa861559116 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:28.177130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ff57c-700a-41c5-90fc-dfa861559116 None admin] PATH_INFO: `/v3/users/ada63f4d813a4591a86492608c197e26` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:28.183669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ff57c-700a-41c5-90fc-dfa861559116 None admin] RBAC: Authorizing `identity:get_user(user_id=ada63f4d813a4591a86492608c197e26)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:28.187631 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ff57c-700a-41c5-90fc-dfa861559116 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:28.187631 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 133/261] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:27 2022] GET /identity/v3/users/ada63f4d813a4591a86492608c197e26 => generated 283 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:28.195810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ff57c-700a-41c5-90fc-dfa861559116 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:28.250350 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a65d3-c178-4565-aee0-65a114c5fd9b None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:28.314713 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6bbe9-5b4f-4ef8-a645-b9c3e55f207c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:28.315431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6bbe9-5b4f-4ef8-a645-b9c3e55f207c None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:28.315722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6bbe9-5b4f-4ef8-a645-b9c3e55f207c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:28.316003 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6bbe9-5b4f-4ef8-a645-b9c3e55f207c None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:28.316771 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6bbe9-5b4f-4ef8-a645-b9c3e55f207c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:28.317082 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6bbe9-5b4f-4ef8-a645-b9c3e55f207c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:28.318481 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6bbe9-5b4f-4ef8-a645-b9c3e55f207c None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:28.337156 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 129/262] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:28.343269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6bbe9-5b4f-4ef8-a645-b9c3e55f207c None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:28.411255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:28.415279 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ff57c-700a-41c5-90fc-dfa861559116 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:28.426324 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:28.477997 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05124603-83bf-42d3-ade2-f919c26cae34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:28.479234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05124603-83bf-42d3-ade2-f919c26cae34 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:28.479886 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05124603-83bf-42d3-ade2-f919c26cae34 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:28.480368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05124603-83bf-42d3-ade2-f919c26cae34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:28.564229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6bbe9-5b4f-4ef8-a645-b9c3e55f207c None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:28.584671 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05124603-83bf-42d3-ade2-f919c26cae34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:28.586092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05124603-83bf-42d3-ade2-f919c26cae34 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:28.646640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7d65a-ae9b-48ee-936c-b696a4dd061f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:28.647598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7d65a-ae9b-48ee-936c-b696a4dd061f None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:28.647948 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7d65a-ae9b-48ee-936c-b696a4dd061f None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:28.648228 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7d65a-ae9b-48ee-936c-b696a4dd061f None admin] PATH_INFO: `/v3/users/ada63f4d813a4591a86492608c197e26` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:28.649548 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7d65a-ae9b-48ee-936c-b696a4dd061f None admin] RBAC: Authorizing `identity:get_user(user_id=ada63f4d813a4591a86492608c197e26)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:28.655136 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7d65a-ae9b-48ee-936c-b696a4dd061f None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:28.657122 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 130/263] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:28 2022] GET /identity/v3/users/ada63f4d813a4591a86492608c197e26 => generated 283 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:28.662830 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7d65a-ae9b-48ee-936c-b696a4dd061f None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:28.692377 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:28.703372 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 134/264] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:03:28.713871 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05124603-83bf-42d3-ade2-f919c26cae34 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:28.843024 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05124603-83bf-42d3-ade2-f919c26cae34 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:28.877802 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fddad10-1fe0-4bbe-b653-c1c502cc012b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:28.878695 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddad10-1fe0-4bbe-b653-c1c502cc012b None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:28.879011 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddad10-1fe0-4bbe-b653-c1c502cc012b None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:28.879297 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddad10-1fe0-4bbe-b653-c1c502cc012b None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:28.883689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7d65a-ae9b-48ee-936c-b696a4dd061f None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:28.885909 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fddad10-1fe0-4bbe-b653-c1c502cc012b None admin] RBAC: Authorizing `identity:get_project(project_id=c0336e9e466e4bd4a28254482978f7b2)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:28.887252 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:28.887498 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fddad10-1fe0-4bbe-b653-c1c502cc012b None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:28.895774 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 135/265] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:28 2022] GET /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:28.897169 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f7d10-b2e4-4f5a-ac7d-7296f1ab2840 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:28.897465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f7d10-b2e4-4f5a-ac7d-7296f1ab2840 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:28.897749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f7d10-b2e4-4f5a-ac7d-7296f1ab2840 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:28.899122 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 136/266] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:28.899827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f7d10-b2e4-4f5a-ac7d-7296f1ab2840 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:28.949229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7aa72-0375-4389-b0c1-0a7b1cbd3b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:28.950457 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7aa72-0375-4389-b0c1-0a7b1cbd3b39 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:28.950756 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7aa72-0375-4389-b0c1-0a7b1cbd3b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:28.951100 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7aa72-0375-4389-b0c1-0a7b1cbd3b39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:29.028639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c7aa72-0375-4389-b0c1-0a7b1cbd3b39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:29.030152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c7aa72-0375-4389-b0c1-0a7b1cbd3b39 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:29.037424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:29.079603 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f7d10-b2e4-4f5a-ac7d-7296f1ab2840 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:29.148583 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f144ff-689a-4627-8bfb-878f52e7c58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:29.149172 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f144ff-689a-4627-8bfb-878f52e7c58a None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:29.149461 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f144ff-689a-4627-8bfb-878f52e7c58a None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:29.149740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f144ff-689a-4627-8bfb-878f52e7c58a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:29.150591 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f144ff-689a-4627-8bfb-878f52e7c58a None admin] RBAC: Adding query filter params (role.id=478b327fd22845cc98a2fafe821b1e85, scope.project.id=c0336e9e466e4bd4a28254482978f7b2, user.id=ada63f4d813a4591a86492608c197e26) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:29.150923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f144ff-689a-4627-8bfb-878f52e7c58a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:29.152380 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f144ff-689a-4627-8bfb-878f52e7c58a None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:29.167590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 137/267] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:03:28 2022] GET /identity/v3/role_assignments?user.id=ada63f4d813a4591a86492608c197e26&scope.project.id=c0336e9e466e4bd4a28254482978f7b2&role.id=478b327fd22845cc98a2fafe821b1e85 => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:29.168825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd05b4-6779-4b77-9156-14482efc5e51 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:29.169121 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd05b4-6779-4b77-9156-14482efc5e51 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:29.169402 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd05b4-6779-4b77-9156-14482efc5e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:29.175674 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 131/268] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:03:29.176833 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7aa72-0375-4389-b0c1-0a7b1cbd3b39 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:29.179178 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:29.214968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-a9cd05b4-6779-4b77-9156-14482efc5e51 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:29.214968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-a9cd05b4-6779-4b77-9156-14482efc5e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:29.385591 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7aa72-0375-4389-b0c1-0a7b1cbd3b39 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:29.420433 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 138/269] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:29.422816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0baaa-2c4d-47c9-a24e-744515c83a4b None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:29.422816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0baaa-2c4d-47c9-a24e-744515c83a4b None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:29.423106 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0baaa-2c4d-47c9-a24e-744515c83a4b None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:29.427183 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 139/270] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:29.443288 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fe76f-5878-4f4c-a5be-081b09a6cc00 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:29.443589 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fe76f-5878-4f4c-a5be-081b09a6cc00 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:29.443845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fe76f-5878-4f4c-a5be-081b09a6cc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:29.445432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d545be4-5b79-4b84-a328-feaee8f4cfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:29.446303 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d545be4-5b79-4b84-a328-feaee8f4cfa5 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:29.448413 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d545be4-5b79-4b84-a328-feaee8f4cfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:29.452711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d545be4-5b79-4b84-a328-feaee8f4cfa5 None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:29.458781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d545be4-5b79-4b84-a328-feaee8f4cfa5 None admin] RBAC: Authorizing `identity:get_project(project_id=c0336e9e466e4bd4a28254482978f7b2)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:29.460305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d545be4-5b79-4b84-a328-feaee8f4cfa5 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:29.462211 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 132/271] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:29 2022] GET /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2 => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:29.462865 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d545be4-5b79-4b84-a328-feaee8f4cfa5 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:29.467782 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-2a3fe76f-5878-4f4c-a5be-081b09a6cc00 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:29.469068 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-2a3fe76f-5878-4f4c-a5be-081b09a6cc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:29.536706 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:29.550095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 140/272] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:03:29.555062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe197df0-a516-4fa7-81df-4d1bf00231de None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:29.555062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe197df0-a516-4fa7-81df-4d1bf00231de None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:29.555062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe197df0-a516-4fa7-81df-4d1bf00231de None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:29.555062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 141/273] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:29.555062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe197df0-a516-4fa7-81df-4d1bf00231de None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:29.556987 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:29.602829 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d545be4-5b79-4b84-a328-feaee8f4cfa5 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:29.641644 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b430e7-cc73-4c60-a161-916faebcc1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:29.642447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b430e7-cc73-4c60-a161-916faebcc1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:29.642802 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b430e7-cc73-4c60-a161-916faebcc1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:29.643135 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b430e7-cc73-4c60-a161-916faebcc1f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:29.654140 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b430e7-cc73-4c60-a161-916faebcc1f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:29.662265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-25b430e7-cc73-4c60-a161-916faebcc1f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:29.662265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:29.662265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:29.662265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:29.662265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:29.662265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:29.662265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:29.662265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:29.662265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:29.662265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:29.662265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:29.662265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:29.662265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:29.662265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:29.662265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:29.662265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:29.663432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:29.663432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:29.663432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:29.663432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:29.663432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:29.663432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:29.663432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:29.663432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:29.663432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:29.663432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:29.663432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:29.663432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:29.663432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:29.663432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:29.663432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:29.663432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:29.664472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:29.664472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:29.664472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:29.664472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:29.664472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:29.664472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:29.664472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:29.664472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:29.664472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:29.664472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:29.664472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:29.664472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:29.664472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 133/274] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:29.665340 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc42670-7938-4046-8592-996aa318cc73 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:29.665340 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc42670-7938-4046-8592-996aa318cc73 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:29.665340 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc42670-7938-4046-8592-996aa318cc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:29.686663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-afc42670-7938-4046-8592-996aa318cc73 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:29.687858 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-afc42670-7938-4046-8592-996aa318cc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:29.688438 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe197df0-a516-4fa7-81df-4d1bf00231de None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:29.725876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630f822-e53c-4192-bf2c-b773590876ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:29.726535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630f822-e53c-4192-bf2c-b773590876ff None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:29.726819 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630f822-e53c-4192-bf2c-b773590876ff None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:29.727119 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630f822-e53c-4192-bf2c-b773590876ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:29.727945 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630f822-e53c-4192-bf2c-b773590876ff None admin] RBAC: Adding query filter params (role.id=9cd080b50c6f4ee8a54bf9ef1655bd83, scope.project.id=c0336e9e466e4bd4a28254482978f7b2, user.id=ada63f4d813a4591a86492608c197e26) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:29.728256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630f822-e53c-4192-bf2c-b773590876ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:29.729683 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630f822-e53c-4192-bf2c-b773590876ff None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:29.739067 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 142/275] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:03:29 2022] GET /identity/v3/role_assignments?user.id=ada63f4d813a4591a86492608c197e26&scope.project.id=c0336e9e466e4bd4a28254482978f7b2&role.id=9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:29.739985 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630f822-e53c-4192-bf2c-b773590876ff None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:29.769888 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 134/276] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:29.771188 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc42670-7938-4046-8592-996aa318cc73 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:29.829647 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:29.855759 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:29.883110 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630f822-e53c-4192-bf2c-b773590876ff None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:29.921173 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc42670-7938-4046-8592-996aa318cc73 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:29.948706 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a4df13-ff6c-460f-9c50-d63dec4e2604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:29.949763 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4df13-ff6c-460f-9c50-d63dec4e2604 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:29.950058 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4df13-ff6c-460f-9c50-d63dec4e2604 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:29.950345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4df13-ff6c-460f-9c50-d63dec4e2604 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:29.952184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a4df13-ff6c-460f-9c50-d63dec4e2604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:29.953942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a4df13-ff6c-460f-9c50-d63dec4e2604 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:29.956262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 143/277] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:03:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:29.956965 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a4df13-ff6c-460f-9c50-d63dec4e2604 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:29.965260 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874f077-25e4-4766-9f8e-8b6fa63351e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:29.966464 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874f077-25e4-4766-9f8e-8b6fa63351e0 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:29.966756 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874f077-25e4-4766-9f8e-8b6fa63351e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:29.967145 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874f077-25e4-4766-9f8e-8b6fa63351e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:29.967930 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874f077-25e4-4766-9f8e-8b6fa63351e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:29.968273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874f077-25e4-4766-9f8e-8b6fa63351e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:29.969683 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874f077-25e4-4766-9f8e-8b6fa63351e0 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:29.979717 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 135/278] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:29.980531 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874f077-25e4-4766-9f8e-8b6fa63351e0 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:30.085517 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a4df13-ff6c-460f-9c50-d63dec4e2604 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:30.106500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874f077-25e4-4766-9f8e-8b6fa63351e0 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:30.111772 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:30.124200 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b146a-b496-41a2-a11d-ec942d21a089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:30.125248 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b146a-b496-41a2-a11d-ec942d21a089 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:30.125711 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b146a-b496-41a2-a11d-ec942d21a089 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:30.126165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b146a-b496-41a2-a11d-ec942d21a089 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:30.130570 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:30.136935 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b146a-b496-41a2-a11d-ec942d21a089 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:30.142164 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7c656-00b9-4209-a14f-491013e1c890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:30.142929 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7c656-00b9-4209-a14f-491013e1c890 None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:30.143219 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7c656-00b9-4209-a14f-491013e1c890 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:30.143496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7c656-00b9-4209-a14f-491013e1c890 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:30.144199 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7c656-00b9-4209-a14f-491013e1c890 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:30.145644 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7c656-00b9-4209-a14f-491013e1c890 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:30.146398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-d98b146a-b496-41a2-a11d-ec942d21a089 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 09:03:30.146398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:30.146398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:30.146398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:30.146398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:30.146398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:30.146398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:30.146398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:30.146398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:30.146398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:30.146398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:30.146398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:30.146398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:30.146398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:30.146398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:30.146398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:30.147321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:30.147321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:30.147321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:30.147321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:30.147321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:30.147321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:30.147321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:30.147321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:30.147321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:30.147321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:30.147321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:30.147321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:30.147321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:30.147321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:30.147321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:30.147321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:30.149035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:30.149035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:30.149035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:30.149035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:30.149035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:30.149035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:30.149035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:30.149035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:30.149035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:30.149035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:30.149035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 09:03:30.149035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:30.149035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 144/279] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:30.149035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b146a-b496-41a2-a11d-ec942d21a089 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:30.265371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b146a-b496-41a2-a11d-ec942d21a089 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:30.281830 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 136/280] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jun 23 09:03:29 2022] POST /identity/v3/users => generated 293 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:03:30.282676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7c656-00b9-4209-a14f-491013e1c890 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:30.325537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3ea79-e051-480b-9575-ae163b47c5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:30.326442 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3ea79-e051-480b-9575-ae163b47c5ad None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:30.326724 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3ea79-e051-480b-9575-ae163b47c5ad None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:30.328679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3ea79-e051-480b-9575-ae163b47c5ad None admin] PATH_INFO: `/v3/users/aaabf59180154887bcb7dbf9925cb65b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:30.350782 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3ea79-e051-480b-9575-ae163b47c5ad None admin] RBAC: Authorizing `identity:get_user(user_id=aaabf59180154887bcb7dbf9925cb65b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:30.352547 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3ea79-e051-480b-9575-ae163b47c5ad None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:30.354429 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 145/281] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:30 2022] GET /identity/v3/users/aaabf59180154887bcb7dbf9925cb65b => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:30.360828 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3ea79-e051-480b-9575-ae163b47c5ad None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:30.413014 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7c656-00b9-4209-a14f-491013e1c890 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:30.420401 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:30.450939 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c8347-2a73-4bbc-ada5-09848240a9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:30.452013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c8347-2a73-4bbc-ada5-09848240a9ab None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:30.452519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c8347-2a73-4bbc-ada5-09848240a9ab None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:30.453040 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c8347-2a73-4bbc-ada5-09848240a9ab None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:30.454068 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c8347-2a73-4bbc-ada5-09848240a9ab None admin] RBAC: Adding query filter params (name=reader) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:30.454385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c8347-2a73-4bbc-ada5-09848240a9ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:30.455813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c8347-2a73-4bbc-ada5-09848240a9ab None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:30.466337 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 137/282] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:30.467997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63f89f-395b-4726-9f41-35c81b492bdb None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:30.468464 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63f89f-395b-4726-9f41-35c81b492bdb None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:30.468925 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63f89f-395b-4726-9f41-35c81b492bdb None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:30.474730 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 138/283] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:30.474730 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63f89f-395b-4726-9f41-35c81b492bdb None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:30.505430 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3ea79-e051-480b-9575-ae163b47c5ad None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:30.524722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:30.551273 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea7868-a299-4f0b-8e52-587899aa5dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:30.551993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea7868-a299-4f0b-8e52-587899aa5dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:30.552284 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea7868-a299-4f0b-8e52-587899aa5dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:30.552571 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea7868-a299-4f0b-8e52-587899aa5dc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:30.553429 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea7868-a299-4f0b-8e52-587899aa5dc2 None admin] RBAC: Adding query filter params (role.id=53a3d65343394e7caafcc33f12cf138b, user.id=aaabf59180154887bcb7dbf9925cb65b, scope.system=all) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:30.553843 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea7868-a299-4f0b-8e52-587899aa5dc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:30.555279 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea7868-a299-4f0b-8e52-587899aa5dc2 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:30.563338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 146/284] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Jun 23 09:03:30 2022] GET /identity/v3/role_assignments?user.id=aaabf59180154887bcb7dbf9925cb65b&scope.system=all&role.id=53a3d65343394e7caafcc33f12cf138b => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:30.564814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e4e49-7f5d-41dc-8d66-abae9139574a None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:30.565113 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e4e49-7f5d-41dc-8d66-abae9139574a None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:30.565402 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e4e49-7f5d-41dc-8d66-abae9139574a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:30.584469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-ae0e4e49-7f5d-41dc-8d66-abae9139574a None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:30.585780 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-ae0e4e49-7f5d-41dc-8d66-abae9139574a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:30.598848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63f89f-395b-4726-9f41-35c81b492bdb None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:30.650734 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510eb7a-20bb-4add-a305-dfcd5109c64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:30.651710 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510eb7a-20bb-4add-a305-dfcd5109c64c None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:30.652003 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510eb7a-20bb-4add-a305-dfcd5109c64c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:30.652289 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510eb7a-20bb-4add-a305-dfcd5109c64c None admin] PATH_INFO: `/v3/users/788b5a145ce24c9999b6880a95294f99` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:30.653624 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0510eb7a-20bb-4add-a305-dfcd5109c64c None admin] RBAC: Authorizing `identity:get_user(user_id=788b5a145ce24c9999b6880a95294f99)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:30.655176 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0510eb7a-20bb-4add-a305-dfcd5109c64c None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:30.656945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 139/285] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:30 2022] GET /identity/v3/users/788b5a145ce24c9999b6880a95294f99 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:30.662744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510eb7a-20bb-4add-a305-dfcd5109c64c None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:30.680283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 147/286] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:30.704464 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e4e49-7f5d-41dc-8d66-abae9139574a None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:30.763326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:30.786673 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510eb7a-20bb-4add-a305-dfcd5109c64c None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:30.797270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:30.825682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d892c-2c00-43ad-8853-7428d0820724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:30.826697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d892c-2c00-43ad-8853-7428d0820724 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:30.827115 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d892c-2c00-43ad-8853-7428d0820724 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:30.827496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d892c-2c00-43ad-8853-7428d0820724 None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:30.829053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d892c-2c00-43ad-8853-7428d0820724 None admin] RBAC: Authorizing `identity:get_project(project_id=c0336e9e466e4bd4a28254482978f7b2)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:30.830845 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d892c-2c00-43ad-8853-7428d0820724 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:30.833116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 140/287] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:30 2022] GET /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:30.845337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e4e49-7f5d-41dc-8d66-abae9139574a None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:30.848837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d892c-2c00-43ad-8853-7428d0820724 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:30.885624 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ef552-5e9e-4f65-96ee-fda52d1af5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:30.886431 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ef552-5e9e-4f65-96ee-fda52d1af5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:30.886722 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ef552-5e9e-4f65-96ee-fda52d1af5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:30.887042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ef552-5e9e-4f65-96ee-fda52d1af5e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:30.899598 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ef552-5e9e-4f65-96ee-fda52d1af5e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:30.908495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-e96ef552-5e9e-4f65-96ee-fda52d1af5e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:30.908495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:30.908495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:30.908495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:30.908495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:30.908495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:30.908495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:30.908495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:30.908495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:30.908495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:30.908495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:30.908495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:30.908495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:30.908495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:30.908495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:30.908495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:30.909467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:30.909467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:30.909467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:30.909467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:30.909467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:30.909467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:30.909467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:30.909467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:30.909467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:30.909467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:30.909467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:30.909467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:30.909467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:30.909467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:30.909467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:30.909467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:30.910393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:30.910393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:30.910393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:30.910393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:30.910393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:30.910393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:30.910393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:30.910393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:30.910393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:30.910393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:30.910393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:30.910393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:30.910393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 148/288] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:03:30.919131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ef552-5e9e-4f65-96ee-fda52d1af5e4 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:31.043455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:31.049205 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ef552-5e9e-4f65-96ee-fda52d1af5e4 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:31.085699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0728ad-0d93-4997-8e22-2ec4407527b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:31.086547 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0728ad-0d93-4997-8e22-2ec4407527b8 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:31.086846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0728ad-0d93-4997-8e22-2ec4407527b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:31.087163 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0728ad-0d93-4997-8e22-2ec4407527b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:31.087931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0728ad-0d93-4997-8e22-2ec4407527b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:31.088364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0728ad-0d93-4997-8e22-2ec4407527b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:31.089793 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0728ad-0d93-4997-8e22-2ec4407527b8 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:31.100174 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 149/289] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:31.105957 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0728ad-0d93-4997-8e22-2ec4407527b8 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:31.141726 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d892c-2c00-43ad-8853-7428d0820724 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:31.192945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424158b7-dbb3-4fef-87cf-04651f582f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:31.193891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424158b7-dbb3-4fef-87cf-04651f582f70 None admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:31.194178 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424158b7-dbb3-4fef-87cf-04651f582f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:31.194458 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424158b7-dbb3-4fef-87cf-04651f582f70 None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2/users/788b5a145ce24c9999b6880a95294f99/roles/a6f4585744cf42e6b971891968972799` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:31.199425 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424158b7-dbb3-4fef-87cf-04651f582f70 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0336e9e466e4bd4a28254482978f7b2, user_id=788b5a145ce24c9999b6880a95294f99, role_id=a6f4585744cf42e6b971891968972799)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:31.200961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424158b7-dbb3-4fef-87cf-04651f582f70 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:31.260090 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0728ad-0d93-4997-8e22-2ec4407527b8 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:31.290569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 141/290] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:03:30 2022] PUT /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2/users/788b5a145ce24c9999b6880a95294f99/roles/a6f4585744cf42e6b971891968972799 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:03:31.305074 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c631e1-55f6-449e-98ec-1ddddf51e0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:31.306010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c631e1-55f6-449e-98ec-1ddddf51e0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:31.306304 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c631e1-55f6-449e-98ec-1ddddf51e0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:31.306582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c631e1-55f6-449e-98ec-1ddddf51e0d7 None admin] PATH_INFO: `/v3/users/942cda9cd07c437db4064f34a91e2c70` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:31.307830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c631e1-55f6-449e-98ec-1ddddf51e0d7 None admin] RBAC: Authorizing `identity:get_user(user_id=942cda9cd07c437db4064f34a91e2c70)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:31.309908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c631e1-55f6-449e-98ec-1ddddf51e0d7 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:31.312002 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 150/291] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:31 2022] GET /identity/v3/users/942cda9cd07c437db4064f34a91e2c70 => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:31.317762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424158b7-dbb3-4fef-87cf-04651f582f70 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:31.363586 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:31.460232 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424158b7-dbb3-4fef-87cf-04651f582f70 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:31.513454 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdef2d5-3b07-4cc6-a2a6-75657f8c68ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:31.514489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdef2d5-3b07-4cc6-a2a6-75657f8c68ae None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:31.515036 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdef2d5-3b07-4cc6-a2a6-75657f8c68ae None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:31.515511 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdef2d5-3b07-4cc6-a2a6-75657f8c68ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:31.589989 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdef2d5-3b07-4cc6-a2a6-75657f8c68ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:31.591914 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdef2d5-3b07-4cc6-a2a6-75657f8c68ae None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:31.627127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:31.718370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 142/292] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:03:31.724496 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c631e1-55f6-449e-98ec-1ddddf51e0d7 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:31.771273 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:31.879520 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c631e1-55f6-449e-98ec-1ddddf51e0d7 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:31.924002 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23140516-a149-425c-8ff7-d960e63af9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:31.924880 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23140516-a149-425c-8ff7-d960e63af9fd None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:31.925175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23140516-a149-425c-8ff7-d960e63af9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:31.925451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23140516-a149-425c-8ff7-d960e63af9fd None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:31.926653 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23140516-a149-425c-8ff7-d960e63af9fd None admin] RBAC: Authorizing `identity:get_project(project_id=c0336e9e466e4bd4a28254482978f7b2)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:31.928122 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23140516-a149-425c-8ff7-d960e63af9fd None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:31.930107 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 151/293] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:31 2022] GET /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:31.936414 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdef2d5-3b07-4cc6-a2a6-75657f8c68ae None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:32.087111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdef2d5-3b07-4cc6-a2a6-75657f8c68ae None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:32.141077 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465acfe-d301-46f8-a5dc-286898cb9157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:32.142082 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465acfe-d301-46f8-a5dc-286898cb9157 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:32.142571 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465acfe-d301-46f8-a5dc-286898cb9157 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:32.143087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465acfe-d301-46f8-a5dc-286898cb9157 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:32.144125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465acfe-d301-46f8-a5dc-286898cb9157 None admin] RBAC: Adding query filter params (role.id=53a3d65343394e7caafcc33f12cf138b, scope.project.id=c0336e9e466e4bd4a28254482978f7b2, user.id=942cda9cd07c437db4064f34a91e2c70) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:32.144639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465acfe-d301-46f8-a5dc-286898cb9157 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:32.146283 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465acfe-d301-46f8-a5dc-286898cb9157 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:32.152231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:32.160557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 143/294] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:03:31 2022] GET /identity/v3/role_assignments?user.id=942cda9cd07c437db4064f34a91e2c70&scope.project.id=c0336e9e466e4bd4a28254482978f7b2&role.id=53a3d65343394e7caafcc33f12cf138b => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:32.924899 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae148857-d896-45ee-895e-ef84049c576d None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:32.925414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae148857-d896-45ee-895e-ef84049c576d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:32.925493 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae148857-d896-45ee-895e-ef84049c576d None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:32.926980 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 152/295] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:32.931989 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c1aa5-4eb7-4c55-b369-1f04e98990b1 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:32.932381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c1aa5-4eb7-4c55-b369-1f04e98990b1 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:32.932548 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c1aa5-4eb7-4c55-b369-1f04e98990b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:32.947121 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be7a82-5c2c-4752-a472-6f0af8eab53d None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:32.947415 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be7a82-5c2c-4752-a472-6f0af8eab53d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:32.947698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be7a82-5c2c-4752-a472-6f0af8eab53d None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:32.949324 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 153/296] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:03:32.950071 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-098c1aa5-4eb7-4c55-b369-1f04e98990b1 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:32.951358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-098c1aa5-4eb7-4c55-b369-1f04e98990b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:32.954323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b8d85-020e-410c-ad62-96536e3c8b09 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:32.954620 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b8d85-020e-410c-ad62-96536e3c8b09 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:32.954923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b8d85-020e-410c-ad62-96536e3c8b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:32.972455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-ae2b8d85-020e-410c-ad62-96536e3c8b09 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:32.973640 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-ae2b8d85-020e-410c-ad62-96536e3c8b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:33.031186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 144/297] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:33.032654 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e8749-2535-4230-b6f2-355e5f4f0634 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:33.032955 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e8749-2535-4230-b6f2-355e5f4f0634 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:33.033239 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e8749-2535-4230-b6f2-355e5f4f0634 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:33.034676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 145/298] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:33.042653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937619f0-bdb4-4409-80d3-942001ea1914 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:33.042653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937619f0-bdb4-4409-80d3-942001ea1914 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:33.042653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937619f0-bdb4-4409-80d3-942001ea1914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:33.050061 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 154/299] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:03:33.056258 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b8d85-020e-410c-ad62-96536e3c8b09 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:33.061207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-937619f0-bdb4-4409-80d3-942001ea1914 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:33.062409 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-937619f0-bdb4-4409-80d3-942001ea1914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:33.080854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:33.139261 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 146/300] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:03:33.139261 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937619f0-bdb4-4409-80d3-942001ea1914 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:33.165638 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b8d85-020e-410c-ad62-96536e3c8b09 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:33.206569 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517b8ea-8aef-4f6d-9b89-2375227edba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:33.208181 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517b8ea-8aef-4f6d-9b89-2375227edba2 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:33.208478 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517b8ea-8aef-4f6d-9b89-2375227edba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:33.208756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517b8ea-8aef-4f6d-9b89-2375227edba2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:33.217008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:33.221374 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517b8ea-8aef-4f6d-9b89-2375227edba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:33.229412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-9517b8ea-8aef-4f6d-9b89-2375227edba2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 09:03:33.229412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:33.229412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:33.229412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:33.229412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:33.229412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:33.229412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:33.229412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:33.229412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:33.229412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:33.229412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:33.229412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:33.229412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:33.229412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:33.229412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:33.229412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:33.230364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:33.230364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:33.230364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:33.230364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:33.230364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:33.230364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:33.230364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:33.230364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:33.230364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:33.230364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:33.230364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:33.230364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:33.230364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:33.230364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:33.230364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:33.230364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:33.231784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:33.231784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:33.231784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:33.231784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:33.231784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:33.231784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:33.231784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:33.231784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:33.231784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:33.231784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:33.231784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 09:03:33.231784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:33.231784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 155/301] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:03:33.234882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517b8ea-8aef-4f6d-9b89-2375227edba2 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:33.281403 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937619f0-bdb4-4409-80d3-942001ea1914 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:33.321760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd5e45-4be5-4997-8ed3-191c7d28b0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:33.322571 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd5e45-4be5-4997-8ed3-191c7d28b0ad None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:33.322860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd5e45-4be5-4997-8ed3-191c7d28b0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:33.323171 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd5e45-4be5-4997-8ed3-191c7d28b0ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:33.324362 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd5e45-4be5-4997-8ed3-191c7d28b0ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:33.325879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd5e45-4be5-4997-8ed3-191c7d28b0ad None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:33.327569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 147/302] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:03:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:33.328340 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd5e45-4be5-4997-8ed3-191c7d28b0ad None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:33.335328 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:33.353880 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517b8ea-8aef-4f6d-9b89-2375227edba2 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:33.397749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0ee76-35bc-4e99-b5b7-c53e6dd3d4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:33.399133 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0ee76-35bc-4e99-b5b7-c53e6dd3d4fd None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:33.399624 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0ee76-35bc-4e99-b5b7-c53e6dd3d4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:33.400076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0ee76-35bc-4e99-b5b7-c53e6dd3d4fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:33.413678 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a0ee76-35bc-4e99-b5b7-c53e6dd3d4fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:33.421983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-07a0ee76-35bc-4e99-b5b7-c53e6dd3d4fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:33.421983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:33.421983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:33.421983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:33.421983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:33.421983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:33.421983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:33.421983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:33.421983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:33.421983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:33.421983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:33.421983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:33.421983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:33.421983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:33.421983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:33.421983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:33.422931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:33.422931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:33.422931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:33.422931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:33.422931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:33.422931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:33.422931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:33.422931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:33.422931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:33.422931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:33.422931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:33.422931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:33.422931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:33.422931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:33.422931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:33.422931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:33.427882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:33.427882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:33.427882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:33.427882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:33.427882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:33.427882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:33.427882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:33.427882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:33.427882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:33.427882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:33.427882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:33.427882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:33.427882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 156/303] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:33.427882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0ee76-35bc-4e99-b5b7-c53e6dd3d4fd None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:33.442808 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd5e45-4be5-4997-8ed3-191c7d28b0ad None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:33.472220 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:33.480372 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f762b82-3448-4762-bf50-6d1898b44362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:33.481108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f762b82-3448-4762-bf50-6d1898b44362 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:33.481413 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f762b82-3448-4762-bf50-6d1898b44362 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:33.481680 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f762b82-3448-4762-bf50-6d1898b44362 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:33.485472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f762b82-3448-4762-bf50-6d1898b44362 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:33.485472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f762b82-3448-4762-bf50-6d1898b44362 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:33.485472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f762b82-3448-4762-bf50-6d1898b44362 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:33.492542 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 148/304] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:33.498093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f762b82-3448-4762-bf50-6d1898b44362 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:33.531472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0ee76-35bc-4e99-b5b7-c53e6dd3d4fd None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:33.566183 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957c65e-5457-444a-bdbd-40273266c874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:33.566863 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957c65e-5457-444a-bdbd-40273266c874 None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:33.567184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957c65e-5457-444a-bdbd-40273266c874 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:33.567467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957c65e-5457-444a-bdbd-40273266c874 None admin] PATH_INFO: `/v3/groups` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:33.568163 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957c65e-5457-444a-bdbd-40273266c874 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:33.569618 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957c65e-5457-444a-bdbd-40273266c874 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:33.602013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f762b82-3448-4762-bf50-6d1898b44362 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:33.636834 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34adf44-db34-4b0d-ab34-400250d964b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:33.638084 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34adf44-db34-4b0d-ab34-400250d964b1 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:33.638371 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34adf44-db34-4b0d-ab34-400250d964b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:33.638649 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34adf44-db34-4b0d-ab34-400250d964b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:33.639454 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34adf44-db34-4b0d-ab34-400250d964b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:33.639845 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34adf44-db34-4b0d-ab34-400250d964b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:33.641246 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34adf44-db34-4b0d-ab34-400250d964b1 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:33.655348 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 149/305] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:33.655348 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34adf44-db34-4b0d-ab34-400250d964b1 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:33.658554 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 157/306] 10.222.0.52 () {60 vars in 1134 bytes} [Thu Jun 23 09:03:33 2022] POST /identity/v3/groups => generated 229 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:03:33.669061 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957c65e-5457-444a-bdbd-40273266c874 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:33.671301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:33.724317 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:33.778096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34adf44-db34-4b0d-ab34-400250d964b1 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:33.799593 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957c65e-5457-444a-bdbd-40273266c874 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:33.816531 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9714fe9-c80d-45eb-a66b-ae575e3d22d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:33.818020 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9714fe9-c80d-45eb-a66b-ae575e3d22d4 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:33.818020 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9714fe9-c80d-45eb-a66b-ae575e3d22d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:33.818020 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9714fe9-c80d-45eb-a66b-ae575e3d22d4 None admin] PATH_INFO: `/v3/users/9f3f6516e36541a896c9c0098655a852` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:33.835898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9714fe9-c80d-45eb-a66b-ae575e3d22d4 None admin] RBAC: Authorizing `identity:get_user(user_id=9f3f6516e36541a896c9c0098655a852)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:33.837398 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9714fe9-c80d-45eb-a66b-ae575e3d22d4 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:33.839170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 150/307] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:33 2022] GET /identity/v3/users/9f3f6516e36541a896c9c0098655a852 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:33.840558 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af76e3-6aec-4d02-92d8-d4c2bb70c51c None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:33.840852 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af76e3-6aec-4d02-92d8-d4c2bb70c51c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:33.841236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af76e3-6aec-4d02-92d8-d4c2bb70c51c None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:33.842113 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efe9b0-0308-4727-8dab-a47f566d5c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:33.842604 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 151/308] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:33.847628 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efe9b0-0308-4727-8dab-a47f566d5c2b None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:33.847628 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efe9b0-0308-4727-8dab-a47f566d5c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:33.847628 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efe9b0-0308-4727-8dab-a47f566d5c2b None admin] PATH_INFO: `/v3/users/aaabf59180154887bcb7dbf9925cb65b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:33.847628 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efe9b0-0308-4727-8dab-a47f566d5c2b None admin] RBAC: Authorizing `identity:get_user(user_id=aaabf59180154887bcb7dbf9925cb65b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:33.847967 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af76e3-6aec-4d02-92d8-d4c2bb70c51c None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:33.848050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efe9b0-0308-4727-8dab-a47f566d5c2b None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:33.848812 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 158/309] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:33 2022] GET /identity/v3/users/aaabf59180154887bcb7dbf9925cb65b => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:33.850454 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba3d41-a431-4057-baa3-4652a8dfa16c None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:33.850454 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba3d41-a431-4057-baa3-4652a8dfa16c None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:33.850454 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba3d41-a431-4057-baa3-4652a8dfa16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:33.867641 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-82ba3d41-a431-4057-baa3-4652a8dfa16c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:33.868836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-82ba3d41-a431-4057-baa3-4652a8dfa16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:33.935522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:33.940193 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 159/310] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:33.941041 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba3d41-a431-4057-baa3-4652a8dfa16c None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:33.975407 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af76e3-6aec-4d02-92d8-d4c2bb70c51c None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:33.987213 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:34.011037 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89909e59-750c-48d7-b537-a18f044b1398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:34.011733 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89909e59-750c-48d7-b537-a18f044b1398 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:34.012021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89909e59-750c-48d7-b537-a18f044b1398 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:34.012835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89909e59-750c-48d7-b537-a18f044b1398 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:34.013677 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89909e59-750c-48d7-b537-a18f044b1398 None admin] RBAC: Adding query filter params (role.id=a6f4585744cf42e6b971891968972799, user.id=9f3f6516e36541a896c9c0098655a852, scope.system=all) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:34.013997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89909e59-750c-48d7-b537-a18f044b1398 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:34.015481 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89909e59-750c-48d7-b537-a18f044b1398 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:34.020561 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 152/311] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Jun 23 09:03:33 2022] GET /identity/v3/role_assignments?user.id=9f3f6516e36541a896c9c0098655a852&scope.system=all&role.id=a6f4585744cf42e6b971891968972799 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:34.029167 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89909e59-750c-48d7-b537-a18f044b1398 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:34.056393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba3d41-a431-4057-baa3-4652a8dfa16c None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:34.105819 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e4194-7b39-486e-8169-65fa310fb8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:34.108189 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e4194-7b39-486e-8169-65fa310fb8ef None admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:34.108487 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e4194-7b39-486e-8169-65fa310fb8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:34.108487 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e4194-7b39-486e-8169-65fa310fb8ef None admin] PATH_INFO: `/v3/system/users/aaabf59180154887bcb7dbf9925cb65b/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:34.111530 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e4194-7b39-486e-8169-65fa310fb8ef None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=aaabf59180154887bcb7dbf9925cb65b, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:34.113028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e4194-7b39-486e-8169-65fa310fb8ef None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:34.179567 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89909e59-750c-48d7-b537-a18f044b1398 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:34.224610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0217b8-8d15-436c-a082-bbd371939e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:34.225491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0217b8-8d15-436c-a082-bbd371939e71 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:34.225735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0217b8-8d15-436c-a082-bbd371939e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:34.226017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0217b8-8d15-436c-a082-bbd371939e71 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:34.238991 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0217b8-8d15-436c-a082-bbd371939e71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:34.244927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-fd0217b8-8d15-436c-a082-bbd371939e71 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 09:03:34.244927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:34.244927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:34.244927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:34.244927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:34.244927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:34.244927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:34.244927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:34.244927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:34.244927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:34.244927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:34.244927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:34.244927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:34.244927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:34.244927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:34.244927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:34.245921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:34.245921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:34.245921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:34.245921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:34.245921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:34.245921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:34.245921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:34.245921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:34.245921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:34.245921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:34.245921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:34.245921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:34.245921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:34.245921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:34.245921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:34.245921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:34.247490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:34.247490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:34.247490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:34.247490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:34.247490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:34.247490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:34.247490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:34.247490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:34.247490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:34.247490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:34.247490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 09:03:34.247490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:34.247490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 153/312] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:34.255452 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0217b8-8d15-436c-a082-bbd371939e71 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:34.255609 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 160/313] 10.222.0.52 () {58 vars in 1334 bytes} [Thu Jun 23 09:03:33 2022] PUT /identity/v3/system/users/aaabf59180154887bcb7dbf9925cb65b/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:03:34.298360 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:34.375968 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0217b8-8d15-436c-a082-bbd371939e71 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:34.414830 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63264a3d-3624-43b0-bb4e-bff34ed2f4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:34.415546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63264a3d-3624-43b0-bb4e-bff34ed2f4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:34.415835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63264a3d-3624-43b0-bb4e-bff34ed2f4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:34.416118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63264a3d-3624-43b0-bb4e-bff34ed2f4f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:34.416881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63264a3d-3624-43b0-bb4e-bff34ed2f4f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:34.417194 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63264a3d-3624-43b0-bb4e-bff34ed2f4f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:34.418669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63264a3d-3624-43b0-bb4e-bff34ed2f4f2 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:34.427729 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 154/314] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:34.433741 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e4194-7b39-486e-8169-65fa310fb8ef None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:34.506635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:34.546626 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e4194-7b39-486e-8169-65fa310fb8ef None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:34.581397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47afed2-e185-4272-b7c9-d6bf01fb5b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:34.582288 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47afed2-e185-4272-b7c9-d6bf01fb5b26 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:34.582573 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47afed2-e185-4272-b7c9-d6bf01fb5b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:34.583136 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47afed2-e185-4272-b7c9-d6bf01fb5b26 None admin] PATH_INFO: `/v3/users/788b5a145ce24c9999b6880a95294f99` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:34.584374 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47afed2-e185-4272-b7c9-d6bf01fb5b26 None admin] RBAC: Authorizing `identity:get_user(user_id=788b5a145ce24c9999b6880a95294f99)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:34.585830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47afed2-e185-4272-b7c9-d6bf01fb5b26 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:34.587700 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 161/315] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:34 2022] GET /identity/v3/users/788b5a145ce24c9999b6880a95294f99 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:34.593279 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63264a3d-3624-43b0-bb4e-bff34ed2f4f2 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:34.693549 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:34.701073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63264a3d-3624-43b0-bb4e-bff34ed2f4f2 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:34.737004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1562f-7fc8-47a1-a590-88e4f5ad213f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:34.737670 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1562f-7fc8-47a1-a590-88e4f5ad213f None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:34.737962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1562f-7fc8-47a1-a590-88e4f5ad213f None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:34.738247 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1562f-7fc8-47a1-a590-88e4f5ad213f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:34.811916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1562f-7fc8-47a1-a590-88e4f5ad213f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:34.813441 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1562f-7fc8-47a1-a590-88e4f5ad213f None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:34.917592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 155/316] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:03:34.918391 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:34.923984 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47afed2-e185-4272-b7c9-d6bf01fb5b26 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:35.035763 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47afed2-e185-4272-b7c9-d6bf01fb5b26 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:35.071128 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:35.071998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be55c5b-2a32-4f6c-8f4b-6147a963c894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:35.072868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be55c5b-2a32-4f6c-8f4b-6147a963c894 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:35.073164 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be55c5b-2a32-4f6c-8f4b-6147a963c894 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:35.073447 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be55c5b-2a32-4f6c-8f4b-6147a963c894 None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:35.074704 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be55c5b-2a32-4f6c-8f4b-6147a963c894 None admin] RBAC: Authorizing `identity:get_project(project_id=c0336e9e466e4bd4a28254482978f7b2)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:35.076212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be55c5b-2a32-4f6c-8f4b-6147a963c894 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:35.077983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 162/317] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:34 2022] GET /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:35.084063 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1562f-7fc8-47a1-a590-88e4f5ad213f None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:35.199881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1562f-7fc8-47a1-a590-88e4f5ad213f None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:35.236405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de97657-9468-4e3c-898e-fab94129130d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:35.237456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de97657-9468-4e3c-898e-fab94129130d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:35.237923 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de97657-9468-4e3c-898e-fab94129130d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:35.238407 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de97657-9468-4e3c-898e-fab94129130d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:35.239522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de97657-9468-4e3c-898e-fab94129130d None admin] RBAC: Adding query filter params (role.id=a6f4585744cf42e6b971891968972799, scope.project.id=c0336e9e466e4bd4a28254482978f7b2, user.id=788b5a145ce24c9999b6880a95294f99) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:35.239997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de97657-9468-4e3c-898e-fab94129130d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:35.241601 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de97657-9468-4e3c-898e-fab94129130d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:35.255236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 156/318] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:03:35 2022] GET /identity/v3/role_assignments?user.id=788b5a145ce24c9999b6880a95294f99&scope.project.id=c0336e9e466e4bd4a28254482978f7b2&role.id=a6f4585744cf42e6b971891968972799 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:35.772668 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa6589-f68e-4259-a3f6-94dffcc9a2e1 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:35.772969 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa6589-f68e-4259-a3f6-94dffcc9a2e1 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:35.773388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa6589-f68e-4259-a3f6-94dffcc9a2e1 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:35.774809 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 163/319] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:35.780466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae64241-d715-46e8-b2bd-6cda646ecdaf None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:35.780762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae64241-d715-46e8-b2bd-6cda646ecdaf None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:35.781041 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae64241-d715-46e8-b2bd-6cda646ecdaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:35.797956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-0ae64241-d715-46e8-b2bd-6cda646ecdaf None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:35.799174 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-0ae64241-d715-46e8-b2bd-6cda646ecdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:35.857953 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:35.869011 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 157/320] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:35.892539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa6589-f68e-4259-a3f6-94dffcc9a2e1 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:35.996160 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa6589-f68e-4259-a3f6-94dffcc9a2e1 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:36.029954 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f4f0b-5d0e-46d7-855d-24c8b611b0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:36.030724 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f4f0b-5d0e-46d7-855d-24c8b611b0ac None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:36.031039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f4f0b-5d0e-46d7-855d-24c8b611b0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:36.031363 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f4f0b-5d0e-46d7-855d-24c8b611b0ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:36.032512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f4f0b-5d0e-46d7-855d-24c8b611b0ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:36.033961 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f4f0b-5d0e-46d7-855d-24c8b611b0ac None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:36.035818 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 164/321] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:03:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:36.041309 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae64241-d715-46e8-b2bd-6cda646ecdaf None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:36.143606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae64241-d715-46e8-b2bd-6cda646ecdaf None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:36.183355 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebee86-84c9-47f9-a75a-57b472ec57a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:36.184534 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebee86-84c9-47f9-a75a-57b472ec57a9 None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:36.185052 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebee86-84c9-47f9-a75a-57b472ec57a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:36.185500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebee86-84c9-47f9-a75a-57b472ec57a9 None admin] PATH_INFO: `/v3/groups` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:36.186463 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebee86-84c9-47f9-a75a-57b472ec57a9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:36.188184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebee86-84c9-47f9-a75a-57b472ec57a9 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:36.208502 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f809c4-bed6-403d-9a7a-5cc6e4e1b9f0 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:36.208800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f809c4-bed6-403d-9a7a-5cc6e4e1b9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:36.209086 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f809c4-bed6-403d-9a7a-5cc6e4e1b9f0 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:36.210496 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 165/322] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:03:36.216063 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cabfa-e357-4a3b-9976-6f4f2923e2e8 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:36.216431 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cabfa-e357-4a3b-9976-6f4f2923e2e8 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:36.216800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cabfa-e357-4a3b-9976-6f4f2923e2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:36.233420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-fa8cabfa-e357-4a3b-9976-6f4f2923e2e8 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:36.234698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-fa8cabfa-e357-4a3b-9976-6f4f2923e2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:36.301407 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:36.306830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 166/323] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:03:36.329432 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8cabfa-e357-4a3b-9976-6f4f2923e2e8 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:36.423237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8cabfa-e357-4a3b-9976-6f4f2923e2e8 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:36.455852 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419b09d-9252-4545-9cca-537363af2b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:36.456706 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419b09d-9252-4545-9cca-537363af2b78 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:36.456994 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419b09d-9252-4545-9cca-537363af2b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:36.457273 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419b09d-9252-4545-9cca-537363af2b78 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:36.467179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419b09d-9252-4545-9cca-537363af2b78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:36.475100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-9419b09d-9252-4545-9cca-537363af2b78 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 09:03:36.475100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:36.475100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:36.475100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:36.475100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:36.475100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:36.475100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:36.475100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:36.475100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:36.475100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:36.475100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:36.475100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:36.475100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:36.475100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:36.475100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:36.475100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:36.476208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:36.476208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:36.476208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:36.476208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:36.476208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:36.476208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:36.476208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:36.476208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:36.476208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:36.476208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:36.476208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:36.476208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:36.476208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:36.476208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:36.476208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:36.476208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:36.477239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:36.477239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:36.477239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:36.477239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:36.477239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:36.477239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:36.477239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:36.477239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:36.477239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:36.477239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:36.477239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 09:03:36.477239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:36.477239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 167/324] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:03:36.477970 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b19d4-82b4-47ab-b954-c82109ffaabb None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:36.477970 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b19d4-82b4-47ab-b954-c82109ffaabb None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:36.478138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b19d4-82b4-47ab-b954-c82109ffaabb None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:36.479464 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 168/325] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:36.482151 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b19d4-82b4-47ab-b954-c82109ffaabb None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:36.531087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 158/326] 10.222.0.52 () {60 vars in 1134 bytes} [Thu Jun 23 09:03:36 2022] POST /identity/v3/groups => generated 226 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:03:36.531087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65117023-d41b-4ba0-ad10-96540d2df77c None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:36.531087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65117023-d41b-4ba0-ad10-96540d2df77c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:36.531087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65117023-d41b-4ba0-ad10-96540d2df77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:36.543902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:36.546242 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-65117023-d41b-4ba0-ad10-96540d2df77c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:36.547508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-65117023-d41b-4ba0-ad10-96540d2df77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:36.591951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b19d4-82b4-47ab-b954-c82109ffaabb None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:36.607071 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:36.629977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 159/327] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:36.633746 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158cf24f-782e-48c8-9dae-aa479f85a7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:36.634254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158cf24f-782e-48c8-9dae-aa479f85a7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:36.634719 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158cf24f-782e-48c8-9dae-aa479f85a7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:36.635204 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158cf24f-782e-48c8-9dae-aa479f85a7a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:36.636180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158cf24f-782e-48c8-9dae-aa479f85a7a9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:36.636670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158cf24f-782e-48c8-9dae-aa479f85a7a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:36.639036 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158cf24f-782e-48c8-9dae-aa479f85a7a9 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:36.653778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 169/328] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:36.653995 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65117023-d41b-4ba0-ad10-96540d2df77c None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:36.659219 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158cf24f-782e-48c8-9dae-aa479f85a7a9 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:36.763750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65117023-d41b-4ba0-ad10-96540d2df77c None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:36.776684 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158cf24f-782e-48c8-9dae-aa479f85a7a9 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:36.802021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:36.813897 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f66249-1dfe-484e-af9e-ffe162fbe660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:36.817007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f66249-1dfe-484e-af9e-ffe162fbe660 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:36.817007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f66249-1dfe-484e-af9e-ffe162fbe660 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:36.817007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f66249-1dfe-484e-af9e-ffe162fbe660 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:36.817651 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c00c7-8702-487d-b018-170eeb5ee7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:36.818766 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c00c7-8702-487d-b018-170eeb5ee7eb None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:36.818901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c00c7-8702-487d-b018-170eeb5ee7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:36.819130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c00c7-8702-487d-b018-170eeb5ee7eb None admin] PATH_INFO: `/v3/users/9f3f6516e36541a896c9c0098655a852` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:36.820366 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c00c7-8702-487d-b018-170eeb5ee7eb None admin] RBAC: Authorizing `identity:get_user(user_id=9f3f6516e36541a896c9c0098655a852)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:36.821780 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c00c7-8702-487d-b018-170eeb5ee7eb None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:36.823750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 170/329] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:36 2022] GET /identity/v3/users/9f3f6516e36541a896c9c0098655a852 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:36.828900 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f66249-1dfe-484e-af9e-ffe162fbe660 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:36.830193 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c00c7-8702-487d-b018-170eeb5ee7eb None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:36.837917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-62f66249-1dfe-484e-af9e-ffe162fbe660 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:36.837917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:36.837917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:36.837917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:36.837917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:36.837917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:36.837917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:36.837917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:36.837917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:36.837917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:36.837917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:36.837917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:36.837917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:36.837917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:36.837917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:36.837917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:36.838895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:36.838895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:36.838895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:36.838895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:36.838895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:36.838895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:36.838895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:36.838895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:36.838895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:36.838895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:36.838895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:36.838895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:36.838895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:36.838895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:36.838895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:36.838895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:36.839766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:36.839766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:36.839766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:36.839766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:36.839766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:36.839766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:36.839766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:36.839766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:36.839766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:36.839766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:36.839766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:36.839766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:36.839766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 160/330] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:03:36.845266 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f66249-1dfe-484e-af9e-ffe162fbe660 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:36.899139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:36.967391 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c00c7-8702-487d-b018-170eeb5ee7eb None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:36.973433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f66249-1dfe-484e-af9e-ffe162fbe660 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:37.013097 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a88b7b2-cf44-48c8-b5a5-5aa57488b849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:37.014035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88b7b2-cf44-48c8-b5a5-5aa57488b849 None admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:37.014323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88b7b2-cf44-48c8-b5a5-5aa57488b849 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:37.014652 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88b7b2-cf44-48c8-b5a5-5aa57488b849 None admin] PATH_INFO: `/v3/system/users/9f3f6516e36541a896c9c0098655a852/roles/a6f4585744cf42e6b971891968972799` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:37.016190 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a88b7b2-cf44-48c8-b5a5-5aa57488b849 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9f3f6516e36541a896c9c0098655a852, role_id=a6f4585744cf42e6b971891968972799)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:37.017581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a88b7b2-cf44-48c8-b5a5-5aa57488b849 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:37.018398 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20392c37-e40e-48f5-ac07-ae14f37e87a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:37.019176 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20392c37-e40e-48f5-ac07-ae14f37e87a5 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:37.019472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20392c37-e40e-48f5-ac07-ae14f37e87a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:37.019752 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20392c37-e40e-48f5-ac07-ae14f37e87a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:37.020711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20392c37-e40e-48f5-ac07-ae14f37e87a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:37.021425 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20392c37-e40e-48f5-ac07-ae14f37e87a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:37.023060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20392c37-e40e-48f5-ac07-ae14f37e87a5 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:37.032250 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 161/331] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:37.037744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20392c37-e40e-48f5-ac07-ae14f37e87a5 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:37.108749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 171/332] 10.222.0.52 () {58 vars in 1334 bytes} [Thu Jun 23 09:03:36 2022] PUT /identity/v3/system/users/9f3f6516e36541a896c9c0098655a852/roles/a6f4585744cf42e6b971891968972799 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:03:37.163054 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20392c37-e40e-48f5-ac07-ae14f37e87a5 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:37.165796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:37.211659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60951e56-8238-42a1-b83f-192e7bd91bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:37.212884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60951e56-8238-42a1-b83f-192e7bd91bed None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:37.213408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60951e56-8238-42a1-b83f-192e7bd91bed None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:37.213904 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60951e56-8238-42a1-b83f-192e7bd91bed None admin] PATH_INFO: `/v3/users/aaabf59180154887bcb7dbf9925cb65b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:37.216822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60951e56-8238-42a1-b83f-192e7bd91bed None admin] RBAC: Authorizing `identity:get_user(user_id=aaabf59180154887bcb7dbf9925cb65b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:37.218609 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60951e56-8238-42a1-b83f-192e7bd91bed None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:37.220916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 162/333] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:37 2022] GET /identity/v3/users/aaabf59180154887bcb7dbf9925cb65b => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:37.226984 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a88b7b2-cf44-48c8-b5a5-5aa57488b849 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:37.304166 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:37.357874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a88b7b2-cf44-48c8-b5a5-5aa57488b849 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:37.393473 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f0842-d0b4-4b18-bdee-6d49f166e06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:37.394434 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f0842-d0b4-4b18-bdee-6d49f166e06a None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:37.394942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f0842-d0b4-4b18-bdee-6d49f166e06a None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:37.395435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f0842-d0b4-4b18-bdee-6d49f166e06a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:37.396475 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f0842-d0b4-4b18-bdee-6d49f166e06a None admin] RBAC: Adding query filter params (role.id=53a3d65343394e7caafcc33f12cf138b, user.id=aaabf59180154887bcb7dbf9925cb65b, scope.system=all) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:37.396976 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f0842-d0b4-4b18-bdee-6d49f166e06a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:37.398639 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f0842-d0b4-4b18-bdee-6d49f166e06a None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:37.421568 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 172/334] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Jun 23 09:03:37 2022] GET /identity/v3/role_assignments?user.id=aaabf59180154887bcb7dbf9925cb65b&scope.system=all&role.id=53a3d65343394e7caafcc33f12cf138b => generated 510 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:38.893623 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d2ed0-847a-450f-aeb9-e70f487ad058 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:38.894469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d2ed0-847a-450f-aeb9-e70f487ad058 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:38.894939 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d2ed0-847a-450f-aeb9-e70f487ad058 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:38.896755 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 163/335] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:38.903724 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59997291-08dd-442a-a8bd-0153d4232d6b None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:38.904558 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59997291-08dd-442a-a8bd-0153d4232d6b None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:38.905020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59997291-08dd-442a-a8bd-0153d4232d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:38.925573 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-59997291-08dd-442a-a8bd-0153d4232d6b None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:38.927110 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-59997291-08dd-442a-a8bd-0153d4232d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:39.017716 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 173/336] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:39.054896 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d2ed0-847a-450f-aeb9-e70f487ad058 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:39.247491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b71fe-9d01-4484-a8fd-61a3d0734645 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:39.248154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b71fe-9d01-4484-a8fd-61a3d0734645 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:39.248669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b71fe-9d01-4484-a8fd-61a3d0734645 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:39.250490 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 174/337] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:39.255009 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d2ed0-847a-450f-aeb9-e70f487ad058 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:39.257402 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34c837-57b2-4f59-8e4f-2939a6cfb2b6 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:39.257913 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34c837-57b2-4f59-8e4f-2939a6cfb2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:39.258401 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34c837-57b2-4f59-8e4f-2939a6cfb2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:39.297079 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-cd34c837-57b2-4f59-8e4f-2939a6cfb2b6 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:39.301279 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-cd34c837-57b2-4f59-8e4f-2939a6cfb2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:39.305523 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b31028-76ad-40f4-b31c-11ffbaabebb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:39.306355 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b31028-76ad-40f4-b31c-11ffbaabebb5 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:39.306654 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b31028-76ad-40f4-b31c-11ffbaabebb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:39.307176 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b31028-76ad-40f4-b31c-11ffbaabebb5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:39.329532 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b31028-76ad-40f4-b31c-11ffbaabebb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:39.337600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-95b31028-76ad-40f4-b31c-11ffbaabebb5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:39.337600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:39.337600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:39.337600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:39.337600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:39.337600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:39.337600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:39.337600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:39.337600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:39.337600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:39.337600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:39.337600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:39.337600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:39.337600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:39.337600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:39.337600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:39.338578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:39.338578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:39.338578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:39.338578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:39.338578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:39.338578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:39.338578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:39.338578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:39.338578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:39.338578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:39.338578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:39.338578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:39.338578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:39.338578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:39.338578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:39.338578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:39.339447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:39.339447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:39.339447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:39.339447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:39.339447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:39.339447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:39.339447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:39.339447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:39.339447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:39.339447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:39.339447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:39.339447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:39.351172 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 164/338] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:39.357400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b31028-76ad-40f4-b31c-11ffbaabebb5 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:39.379271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:39.403679 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:39.430945 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 175/339] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:03:39.497594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34c837-57b2-4f59-8e4f-2939a6cfb2b6 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:39.509076 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b31028-76ad-40f4-b31c-11ffbaabebb5 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:39.572574 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5e5d9-fcd9-4f63-9990-1f12f4105570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:39.575050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5e5d9-fcd9-4f63-9990-1f12f4105570 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:39.575050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5e5d9-fcd9-4f63-9990-1f12f4105570 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:39.575050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5e5d9-fcd9-4f63-9990-1f12f4105570 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:39.575050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d5e5d9-fcd9-4f63-9990-1f12f4105570 None admin] RBAC: Adding query filter params (name=member) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:39.575050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d5e5d9-fcd9-4f63-9990-1f12f4105570 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:39.576492 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d5e5d9-fcd9-4f63-9990-1f12f4105570 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:39.593636 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 165/340] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:39.616314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5e5d9-fcd9-4f63-9990-1f12f4105570 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:39.644679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34c837-57b2-4f59-8e4f-2939a6cfb2b6 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:39.704782 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa1c79-5c5d-472a-8630-66b100de21d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:39.705541 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa1c79-5c5d-472a-8630-66b100de21d6 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:39.705828 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa1c79-5c5d-472a-8630-66b100de21d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:39.706107 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa1c79-5c5d-472a-8630-66b100de21d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:39.728297 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa1c79-5c5d-472a-8630-66b100de21d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:39.744854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-3cfa1c79-5c5d-472a-8630-66b100de21d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:39.744854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:39.744854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:39.744854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:39.744854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:39.744854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:39.744854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:39.744854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:39.744854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:39.744854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:39.744854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:39.744854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:39.744854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:39.744854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:39.744854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:39.744854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:39.745868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:39.745868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:39.745868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:39.745868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:39.745868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:39.745868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:39.745868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:39.745868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:39.745868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:39.745868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:39.745868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:39.745868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:39.745868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:39.745868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:39.745868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:39.745868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:39.747210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:39.747210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:39.747210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:39.747210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:39.747210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:39.747210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:39.747210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:39.747210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:39.747210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:39.747210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:39.747210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:39.747210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:39.747210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 176/341] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:03:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:03:39.752943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:39.759949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa1c79-5c5d-472a-8630-66b100de21d6 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:39.764780 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5e5d9-fcd9-4f63-9990-1f12f4105570 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:39.802852 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:39.827665 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71e5d9-5940-48d7-bf55-39db2eac5f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:39.828711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71e5d9-5940-48d7-bf55-39db2eac5f69 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:39.829198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71e5d9-5940-48d7-bf55-39db2eac5f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:39.829671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71e5d9-5940-48d7-bf55-39db2eac5f69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:39.904178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa1c79-5c5d-472a-8630-66b100de21d6 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:39.940774 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71e5d9-5940-48d7-bf55-39db2eac5f69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:39.943094 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71e5d9-5940-48d7-bf55-39db2eac5f69 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:39.954413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493c3d1-b99f-4e88-8596-1d1070406de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:39.955182 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493c3d1-b99f-4e88-8596-1d1070406de3 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:39.955422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493c3d1-b99f-4e88-8596-1d1070406de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:39.955700 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493c3d1-b99f-4e88-8596-1d1070406de3 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:39.956475 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493c3d1-b99f-4e88-8596-1d1070406de3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:39.956775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493c3d1-b99f-4e88-8596-1d1070406de3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:39.958439 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493c3d1-b99f-4e88-8596-1d1070406de3 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:39.969053 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 177/342] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:03:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:39.995604 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493c3d1-b99f-4e88-8596-1d1070406de3 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:40.126584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:40.139924 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 166/343] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 09:03:40.143679 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71e5d9-5940-48d7-bf55-39db2eac5f69 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:40.205967 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:40.232968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493c3d1-b99f-4e88-8596-1d1070406de3 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:40.286952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71e5d9-5940-48d7-bf55-39db2eac5f69 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:40.331656 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc4137-d9de-4606-b5ff-bb4c5221447b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:40.332341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc4137-d9de-4606-b5ff-bb4c5221447b None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:40.332640 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc4137-d9de-4606-b5ff-bb4c5221447b None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:40.333192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc4137-d9de-4606-b5ff-bb4c5221447b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:40.339610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15f0c3-efe0-4b87-867d-44059930d1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:40.341072 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15f0c3-efe0-4b87-867d-44059930d1da None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:40.341358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15f0c3-efe0-4b87-867d-44059930d1da None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:40.341632 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15f0c3-efe0-4b87-867d-44059930d1da None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:40.342896 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15f0c3-efe0-4b87-867d-44059930d1da None admin] RBAC: Authorizing `identity:get_project(project_id=79d2a60c71124efc90e0d0b445517d55)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:40.344436 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15f0c3-efe0-4b87-867d-44059930d1da None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:40.346726 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 167/344] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:40 2022] GET /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:40.352902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15f0c3-efe0-4b87-867d-44059930d1da None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:40.438884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:40.466675 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc4137-d9de-4606-b5ff-bb4c5221447b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:40.468594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc4137-d9de-4606-b5ff-bb4c5221447b None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:40.497312 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15f0c3-efe0-4b87-867d-44059930d1da None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:40.530935 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26213ec2-8bd2-4b5b-bafb-e97b43615aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:40.531971 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26213ec2-8bd2-4b5b-bafb-e97b43615aba None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:40.532264 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26213ec2-8bd2-4b5b-bafb-e97b43615aba None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:40.532544 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26213ec2-8bd2-4b5b-bafb-e97b43615aba None admin] PATH_INFO: `/v3/groups/54b2eb315544408990a4259d8a76c132` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:40.541166 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26213ec2-8bd2-4b5b-bafb-e97b43615aba None admin] RBAC: Authorizing `identity:get_group(group_id=54b2eb315544408990a4259d8a76c132)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:40.542779 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26213ec2-8bd2-4b5b-bafb-e97b43615aba None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:40.545389 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 168/345] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jun 23 09:03:40 2022] GET /identity/v3/groups/54b2eb315544408990a4259d8a76c132 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:40.546882 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed47e88-37ce-46f8-8c18-45c231a1ef14 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:40.547424 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed47e88-37ce-46f8-8c18-45c231a1ef14 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:40.547870 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed47e88-37ce-46f8-8c18-45c231a1ef14 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:40.549556 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 169/346] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:40.550822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cdd49-abf2-4649-bbf6-23f7c5494f38 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:40.551318 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cdd49-abf2-4649-bbf6-23f7c5494f38 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:40.551764 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cdd49-abf2-4649-bbf6-23f7c5494f38 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:40.553144 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 170/347] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:40.560555 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4559f11-7ed9-4c46-b405-cbfd3bed51ab None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:40.561129 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4559f11-7ed9-4c46-b405-cbfd3bed51ab None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:40.561575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4559f11-7ed9-4c46-b405-cbfd3bed51ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:40.575737 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:40.584417 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-b4559f11-7ed9-4c46-b405-cbfd3bed51ab None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:40.585539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 178/348] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 09:03:40.586006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-b4559f11-7ed9-4c46-b405-cbfd3bed51ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:40.586833 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5338a5b-0782-4e6a-85da-420234343083 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:40.588212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5338a5b-0782-4e6a-85da-420234343083 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:40.588212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5338a5b-0782-4e6a-85da-420234343083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:40.605095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-a5338a5b-0782-4e6a-85da-420234343083 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:40.606929 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-a5338a5b-0782-4e6a-85da-420234343083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:40.668240 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 171/349] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:40.670329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4559f11-7ed9-4c46-b405-cbfd3bed51ab None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:40.697567 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 179/350] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:40.698527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5338a5b-0782-4e6a-85da-420234343083 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:40.731792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:40.794333 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4559f11-7ed9-4c46-b405-cbfd3bed51ab None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:40.816824 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5338a5b-0782-4e6a-85da-420234343083 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:40.839394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce08503f-4025-44d6-8038-9002227d1a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:40.840065 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08503f-4025-44d6-8038-9002227d1a12 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:40.840421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08503f-4025-44d6-8038-9002227d1a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:40.840705 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08503f-4025-44d6-8038-9002227d1a12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:40.841557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce08503f-4025-44d6-8038-9002227d1a12 None admin] RBAC: Adding query filter params (group.id=54b2eb315544408990a4259d8a76c132, role.id=53a3d65343394e7caafcc33f12cf138b, scope.project.id=79d2a60c71124efc90e0d0b445517d55) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:40.841872 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce08503f-4025-44d6-8038-9002227d1a12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:40.843894 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce08503f-4025-44d6-8038-9002227d1a12 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:40.848275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:40.852037 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 172/351] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jun 23 09:03:40 2022] GET /identity/v3/role_assignments?group.id=54b2eb315544408990a4259d8a76c132&scope.project.id=79d2a60c71124efc90e0d0b445517d55&role.id=53a3d65343394e7caafcc33f12cf138b => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:40.853334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230db32-9057-4bbb-89e0-5523ac615523 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:40.853623 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230db32-9057-4bbb-89e0-5523ac615523 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:40.853901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230db32-9057-4bbb-89e0-5523ac615523 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:40.856335 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 173/352] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:40.862701 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230db32-9057-4bbb-89e0-5523ac615523 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:40.871591 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca082827-fe94-46f7-a260-902179731aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:40.872494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca082827-fe94-46f7-a260-902179731aac None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:40.872780 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca082827-fe94-46f7-a260-902179731aac None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:40.873060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca082827-fe94-46f7-a260-902179731aac None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:40.876527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca082827-fe94-46f7-a260-902179731aac None admin] RBAC: Authorizing `identity:get_project(project_id=79d2a60c71124efc90e0d0b445517d55)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:40.878029 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca082827-fe94-46f7-a260-902179731aac None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:40.885673 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 180/353] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:40 2022] GET /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:40.887123 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca082827-fe94-46f7-a260-902179731aac None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:40.998175 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230db32-9057-4bbb-89e0-5523ac615523 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:41.016642 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca082827-fe94-46f7-a260-902179731aac None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:41.018800 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:41.037769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a535c82-9b31-4b35-b220-105febf9eed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:41.038574 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a535c82-9b31-4b35-b220-105febf9eed1 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:41.038857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a535c82-9b31-4b35-b220-105febf9eed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:41.039162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a535c82-9b31-4b35-b220-105febf9eed1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:41.051367 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a535c82-9b31-4b35-b220-105febf9eed1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:41.063111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-5a535c82-9b31-4b35-b220-105febf9eed1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:41.063111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:41.063111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:41.063111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:41.063111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:41.063111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:41.063111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:41.063111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:41.063111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:41.063111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:41.063111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:41.063111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:41.063111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:41.063111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:41.063111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:41.063111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:41.064138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:41.064138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:41.064138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:41.064138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:41.064138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:41.064138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:41.064138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:41.064138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:41.064138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:41.064138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:41.064138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:41.064138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:41.064138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:41.064138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:41.064138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:41.064138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:41.065022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:41.065022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:41.065022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:41.065022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:41.065022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:41.065022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:41.065022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:41.065022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:41.065022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:41.065022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:41.065022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:41.065022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:41.065022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 174/354] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:41.066254 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbaacc-2e05-4768-afaa-7aa65e013452 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:41.066254 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbaacc-2e05-4768-afaa-7aa65e013452 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:41.066411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbaacc-2e05-4768-afaa-7aa65e013452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:41.067804 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002c825-a8b4-47a7-863a-9c601da2a824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:41.067804 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002c825-a8b4-47a7-863a-9c601da2a824 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:41.075057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002c825-a8b4-47a7-863a-9c601da2a824 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:41.075057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002c825-a8b4-47a7-863a-9c601da2a824 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:41.084055 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4002c825-a8b4-47a7-863a-9c601da2a824 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:41.085514 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-dfbbaacc-2e05-4768-afaa-7aa65e013452 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:41.086710 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-dfbbaacc-2e05-4768-afaa-7aa65e013452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:41.093019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-4002c825-a8b4-47a7-863a-9c601da2a824 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 09:03:41.093019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:41.093019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:41.093019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:41.093019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:41.093019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:41.093019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:41.093019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:41.093019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:41.093019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:41.093019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:41.093019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:41.093019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:41.093019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:41.093019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:41.093019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:41.093993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:41.093993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:41.093993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:41.093993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:41.093993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:41.093993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:41.093993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:41.093993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:41.093993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:41.093993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:41.093993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:41.093993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:41.093993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:41.093993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:41.093993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:41.093993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:41.094862 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:41.094862 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:41.094862 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:41.094862 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:41.094862 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:41.094862 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:41.094862 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:41.094862 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:41.094862 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:41.094862 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:41.094862 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 09:03:41.094862 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:41.094862 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 181/355] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:41.095604 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002c825-a8b4-47a7-863a-9c601da2a824 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:41.159836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:41.174093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 175/356] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:41.175441 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbaacc-2e05-4768-afaa-7aa65e013452 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:41.220323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002c825-a8b4-47a7-863a-9c601da2a824 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:41.257172 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3818f359-3a2a-431f-9b18-6958cdd7d590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:41.257997 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818f359-3a2a-431f-9b18-6958cdd7d590 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:41.258288 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818f359-3a2a-431f-9b18-6958cdd7d590 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:41.258567 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818f359-3a2a-431f-9b18-6958cdd7d590 None admin] PATH_INFO: `/v3/groups/54b2eb315544408990a4259d8a76c132` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:41.261405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3818f359-3a2a-431f-9b18-6958cdd7d590 None admin] RBAC: Authorizing `identity:get_group(group_id=54b2eb315544408990a4259d8a76c132)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:41.264020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3818f359-3a2a-431f-9b18-6958cdd7d590 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:41.266379 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 182/357] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jun 23 09:03:41 2022] GET /identity/v3/groups/54b2eb315544408990a4259d8a76c132 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:41.267205 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3818f359-3a2a-431f-9b18-6958cdd7d590 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:41.292783 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbaacc-2e05-4768-afaa-7aa65e013452 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:41.306090 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:41.346348 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53650d22-f831-4c88-8685-ec8fc1a78ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:41.347446 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53650d22-f831-4c88-8685-ec8fc1a78ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:41.347998 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53650d22-f831-4c88-8685-ec8fc1a78ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:41.348576 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53650d22-f831-4c88-8685-ec8fc1a78ad9 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:41.349525 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53650d22-f831-4c88-8685-ec8fc1a78ad9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:41.350029 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53650d22-f831-4c88-8685-ec8fc1a78ad9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:41.351758 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53650d22-f831-4c88-8685-ec8fc1a78ad9 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:41.361800 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 176/358] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:41.367387 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53650d22-f831-4c88-8685-ec8fc1a78ad9 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:41.392081 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3818f359-3a2a-431f-9b18-6958cdd7d590 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:41.418018 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:41.440629 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78c402-943a-48b3-ae53-6a7ea3f719ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:41.441775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78c402-943a-48b3-ae53-6a7ea3f719ab None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:41.442601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78c402-943a-48b3-ae53-6a7ea3f719ab None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:41.443142 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78c402-943a-48b3-ae53-6a7ea3f719ab None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:41.444338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78c402-943a-48b3-ae53-6a7ea3f719ab None admin] RBAC: Adding query filter params (name=reader) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:41.444971 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78c402-943a-48b3-ae53-6a7ea3f719ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:41.447174 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78c402-943a-48b3-ae53-6a7ea3f719ab None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:41.470896 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 183/359] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:41.472825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78c402-943a-48b3-ae53-6a7ea3f719ab None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:41.514093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53650d22-f831-4c88-8685-ec8fc1a78ad9 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:41.567591 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48164d91-d737-4827-97ef-bb3c24907609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:41.568422 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48164d91-d737-4827-97ef-bb3c24907609 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:41.568709 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48164d91-d737-4827-97ef-bb3c24907609 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:41.568990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48164d91-d737-4827-97ef-bb3c24907609 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:41.586473 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48164d91-d737-4827-97ef-bb3c24907609 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:41.598658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-48164d91-d737-4827-97ef-bb3c24907609 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:41.598658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:41.598658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:41.598658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:41.598658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:41.598658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:41.598658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:41.598658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:41.598658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:41.598658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:41.598658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:41.598658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:41.598658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:41.598658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:41.598658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:41.598658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:41.599659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:41.599659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:41.599659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:41.599659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:41.599659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:41.599659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:41.599659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:41.599659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:41.599659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:41.599659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:41.599659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:41.599659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:41.599659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:41.599659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:41.599659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:41.599659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:41.600499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:41.600499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:41.600499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:41.600499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:41.600499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:41.600499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:41.600499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:41.600499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:41.600499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:41.600499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:41.600499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:41.600499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:41.603447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 177/360] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:03:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:41.604840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48164d91-d737-4827-97ef-bb3c24907609 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:41.616572 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78c402-943a-48b3-ae53-6a7ea3f719ab None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:41.666747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d98c5-0c79-4f98-a34e-f5551366b399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:41.667637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d98c5-0c79-4f98-a34e-f5551366b399 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:41.668174 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d98c5-0c79-4f98-a34e-f5551366b399 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:41.668496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:41.668604 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d98c5-0c79-4f98-a34e-f5551366b399 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:41.669519 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d98c5-0c79-4f98-a34e-f5551366b399 None admin] RBAC: Adding query filter params (group.id=54b2eb315544408990a4259d8a76c132, role.id=478b327fd22845cc98a2fafe821b1e85, scope.project.id=79d2a60c71124efc90e0d0b445517d55) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:41.670221 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d98c5-0c79-4f98-a34e-f5551366b399 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:41.671968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d98c5-0c79-4f98-a34e-f5551366b399 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:41.679625 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 184/361] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jun 23 09:03:41 2022] GET /identity/v3/role_assignments?group.id=54b2eb315544408990a4259d8a76c132&scope.project.id=79d2a60c71124efc90e0d0b445517d55&role.id=478b327fd22845cc98a2fafe821b1e85 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:41.680864 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d98c5-0c79-4f98-a34e-f5551366b399 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:41.733419 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:41.741437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48164d91-d737-4827-97ef-bb3c24907609 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:41.784824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514d1f1-a02d-4c66-a859-c6aa578abdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:41.785892 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514d1f1-a02d-4c66-a859-c6aa578abdf6 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:41.786405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514d1f1-a02d-4c66-a859-c6aa578abdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:41.786981 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514d1f1-a02d-4c66-a859-c6aa578abdf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:41.799618 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d98c5-0c79-4f98-a34e-f5551366b399 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:41.837182 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53212b0c-e538-4710-993e-d7008ada3f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:41.838081 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53212b0c-e538-4710-993e-d7008ada3f6c None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:41.838371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53212b0c-e538-4710-993e-d7008ada3f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:41.838654 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53212b0c-e538-4710-993e-d7008ada3f6c None admin] PATH_INFO: `/v3/users/9f3f6516e36541a896c9c0098655a852` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:41.840010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53212b0c-e538-4710-993e-d7008ada3f6c None admin] RBAC: Authorizing `identity:get_user(user_id=9f3f6516e36541a896c9c0098655a852)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:41.842070 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53212b0c-e538-4710-993e-d7008ada3f6c None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:41.845060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 185/362] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:03:41 2022] GET /identity/v3/users/9f3f6516e36541a896c9c0098655a852 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:41.845888 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53212b0c-e538-4710-993e-d7008ada3f6c None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:41.869384 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514d1f1-a02d-4c66-a859-c6aa578abdf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:41.871244 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514d1f1-a02d-4c66-a859-c6aa578abdf6 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:41.912219 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:41.990500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53212b0c-e538-4710-993e-d7008ada3f6c None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:41.998282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 178/363] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:03:42.002460 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ca79e-c4f6-48f0-a74e-de5b5073bb97 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:42.002460 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ca79e-c4f6-48f0-a74e-de5b5073bb97 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:42.002460 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ca79e-c4f6-48f0-a74e-de5b5073bb97 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:42.002460 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 179/364] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:42.003377 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ca79e-c4f6-48f0-a74e-de5b5073bb97 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:42.014234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:42.042122 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31bd86d-6fcd-474c-89c1-f7dc4fbd3d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:42.047163 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31bd86d-6fcd-474c-89c1-f7dc4fbd3d97 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:42.047163 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31bd86d-6fcd-474c-89c1-f7dc4fbd3d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:42.047163 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31bd86d-6fcd-474c-89c1-f7dc4fbd3d97 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:42.047710 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31bd86d-6fcd-474c-89c1-f7dc4fbd3d97 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:42.048031 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31bd86d-6fcd-474c-89c1-f7dc4fbd3d97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:42.052431 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31bd86d-6fcd-474c-89c1-f7dc4fbd3d97 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:42.065923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 186/365] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:03:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:42.067669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b229e-f84b-4d3b-801a-5b81ffe3f301 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:42.068172 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b229e-f84b-4d3b-801a-5b81ffe3f301 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:42.069152 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b229e-f84b-4d3b-801a-5b81ffe3f301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:42.092195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-fb9b229e-f84b-4d3b-801a-5b81ffe3f301 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:42.093767 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-fb9b229e-f84b-4d3b-801a-5b81ffe3f301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:42.126544 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ca79e-c4f6-48f0-a74e-de5b5073bb97 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:42.175379 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:42.180994 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 187/366] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:42.182717 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b229e-f84b-4d3b-801a-5b81ffe3f301 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:42.185321 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ef623-17c4-40ab-b79d-a920e9f916de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:42.186205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ef623-17c4-40ab-b79d-a920e9f916de None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:42.186492 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ef623-17c4-40ab-b79d-a920e9f916de None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:42.186771 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ef623-17c4-40ab-b79d-a920e9f916de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:42.188415 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0ef623-17c4-40ab-b79d-a920e9f916de None admin] RBAC: Adding query filter params (role.id=a6f4585744cf42e6b971891968972799, user.id=9f3f6516e36541a896c9c0098655a852, scope.system=all) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:42.188415 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0ef623-17c4-40ab-b79d-a920e9f916de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:42.189604 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0ef623-17c4-40ab-b79d-a920e9f916de None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:42.219536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 180/367] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Jun 23 09:03:42 2022] GET /identity/v3/role_assignments?user.id=9f3f6516e36541a896c9c0098655a852&scope.system=all&role.id=a6f4585744cf42e6b971891968972799 => generated 510 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:42.220366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ef623-17c4-40ab-b79d-a920e9f916de None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:42.309047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b229e-f84b-4d3b-801a-5b81ffe3f301 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:42.313388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:42.351915 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983e3c6-fc5a-4d35-9936-57ed5286d9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:42.353184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983e3c6-fc5a-4d35-9936-57ed5286d9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:42.353850 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983e3c6-fc5a-4d35-9936-57ed5286d9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:42.354336 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983e3c6-fc5a-4d35-9936-57ed5286d9a3 None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:42.356523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6983e3c6-fc5a-4d35-9936-57ed5286d9a3 None admin] RBAC: Authorizing `identity:get_project(project_id=c0336e9e466e4bd4a28254482978f7b2)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:42.358397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6983e3c6-fc5a-4d35-9936-57ed5286d9a3 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:42.361365 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 188/368] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:42 2022] GET /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:42.362557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983e3c6-fc5a-4d35-9936-57ed5286d9a3 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:42.470851 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ef623-17c4-40ab-b79d-a920e9f916de None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:42.521142 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983e3c6-fc5a-4d35-9936-57ed5286d9a3 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:42.523515 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7eb80-229f-4212-bf3b-d34adc96a746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:42.525036 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7eb80-229f-4212-bf3b-d34adc96a746 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:42.525036 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7eb80-229f-4212-bf3b-d34adc96a746 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:42.525036 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7eb80-229f-4212-bf3b-d34adc96a746 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:42.569866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5552a8-0f73-40ec-997e-7bd55f6ee39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:42.571043 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5552a8-0f73-40ec-997e-7bd55f6ee39b None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:42.571486 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5552a8-0f73-40ec-997e-7bd55f6ee39b None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:42.571888 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5552a8-0f73-40ec-997e-7bd55f6ee39b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:42.585030 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5552a8-0f73-40ec-997e-7bd55f6ee39b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:42.596562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-de5552a8-0f73-40ec-997e-7bd55f6ee39b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:03:42.596562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:42.596562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:42.596562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:42.596562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:42.596562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:42.596562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:42.596562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:42.596562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:42.596562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:42.596562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:42.596562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:42.596562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:42.596562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:42.596562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:42.596562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:42.597758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:42.597758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:42.597758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:42.597758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:42.597758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:42.597758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:42.597758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:42.597758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:42.597758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:42.597758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:42.597758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:42.597758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:42.597758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:42.597758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:42.597758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:42.597758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:42.598778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:42.598778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:42.598778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:42.598778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:42.598778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:42.598778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:42.598778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:42.598778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:42.598778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:42.598778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:42.598778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:03:42.598778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:42.601785 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 189/369] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:03:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:42.602850 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5552a8-0f73-40ec-997e-7bd55f6ee39b None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:42.618055 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:42.626512 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7eb80-229f-4212-bf3b-d34adc96a746 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:42.628284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7eb80-229f-4212-bf3b-d34adc96a746 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:42.647808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:42.733808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5552a8-0f73-40ec-997e-7bd55f6ee39b None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:42.759345 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 181/370] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:03:42.760804 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7eb80-229f-4212-bf3b-d34adc96a746 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:42.782984 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb20dfe-bd40-4d0f-98b6-050c8b1ec5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:42.783980 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb20dfe-bd40-4d0f-98b6-050c8b1ec5ba None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:42.784307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb20dfe-bd40-4d0f-98b6-050c8b1ec5ba None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:42.784659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb20dfe-bd40-4d0f-98b6-050c8b1ec5ba None admin] PATH_INFO: `/v3/groups/54b2eb315544408990a4259d8a76c132` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:42.785926 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb20dfe-bd40-4d0f-98b6-050c8b1ec5ba None admin] RBAC: Authorizing `identity:get_group(group_id=54b2eb315544408990a4259d8a76c132)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:42.787503 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb20dfe-bd40-4d0f-98b6-050c8b1ec5ba None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:42.789521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 190/371] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jun 23 09:03:42 2022] GET /identity/v3/groups/54b2eb315544408990a4259d8a76c132 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:42.790387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb20dfe-bd40-4d0f-98b6-050c8b1ec5ba None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:42.881015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:42.891101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7eb80-229f-4212-bf3b-d34adc96a746 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:42.907719 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:42.911785 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb20dfe-bd40-4d0f-98b6-050c8b1ec5ba None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:42.932488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870befc6-e31b-496c-8cdf-fd674bf4f65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:42.933173 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870befc6-e31b-496c-8cdf-fd674bf4f65f None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:42.933462 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870befc6-e31b-496c-8cdf-fd674bf4f65f None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:42.933795 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870befc6-e31b-496c-8cdf-fd674bf4f65f None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:42.934639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870befc6-e31b-496c-8cdf-fd674bf4f65f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:42.934972 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870befc6-e31b-496c-8cdf-fd674bf4f65f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:42.940871 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870befc6-e31b-496c-8cdf-fd674bf4f65f None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:42.944898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 182/372] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:42.945724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870befc6-e31b-496c-8cdf-fd674bf4f65f None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:42.954650 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174513f-7f72-4e5b-9733-3378026fb7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:42.955551 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174513f-7f72-4e5b-9733-3378026fb7a5 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:42.956263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174513f-7f72-4e5b-9733-3378026fb7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:42.956634 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174513f-7f72-4e5b-9733-3378026fb7a5 None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:42.958130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174513f-7f72-4e5b-9733-3378026fb7a5 None admin] RBAC: Authorizing `identity:get_project(project_id=c0336e9e466e4bd4a28254482978f7b2)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:42.960676 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174513f-7f72-4e5b-9733-3378026fb7a5 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:42.962701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 191/373] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:42 2022] GET /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:42.963660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174513f-7f72-4e5b-9733-3378026fb7a5 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:43.059923 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870befc6-e31b-496c-8cdf-fd674bf4f65f None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:43.099351 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174513f-7f72-4e5b-9733-3378026fb7a5 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:43.105384 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2e043-6142-404c-88e2-9c0a87e6e970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:43.106867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2e043-6142-404c-88e2-9c0a87e6e970 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:43.107435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2e043-6142-404c-88e2-9c0a87e6e970 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:43.107923 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2e043-6142-404c-88e2-9c0a87e6e970 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:43.108956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2e043-6142-404c-88e2-9c0a87e6e970 None admin] RBAC: Adding query filter params (group.id=54b2eb315544408990a4259d8a76c132, role.id=53a3d65343394e7caafcc33f12cf138b, scope.project.id=c0336e9e466e4bd4a28254482978f7b2) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:43.109469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2e043-6142-404c-88e2-9c0a87e6e970 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:43.111268 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2e043-6142-404c-88e2-9c0a87e6e970 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:43.122193 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 183/374] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jun 23 09:03:42 2022] GET /identity/v3/role_assignments?group.id=54b2eb315544408990a4259d8a76c132&scope.project.id=c0336e9e466e4bd4a28254482978f7b2&role.id=53a3d65343394e7caafcc33f12cf138b => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:43.123726 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2e043-6142-404c-88e2-9c0a87e6e970 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:43.140331 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0281315-00bd-490e-aedd-6cecd2216ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:43.141296 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0281315-00bd-490e-aedd-6cecd2216ecd None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:43.141602 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0281315-00bd-490e-aedd-6cecd2216ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:43.141869 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0281315-00bd-490e-aedd-6cecd2216ecd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:43.159295 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:43.200251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:43.238784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0281315-00bd-490e-aedd-6cecd2216ecd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:43.241360 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0281315-00bd-490e-aedd-6cecd2216ecd None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:43.296309 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2e043-6142-404c-88e2-9c0a87e6e970 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:43.354262 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec56cce-7836-4cf4-a5bd-dfcabb478b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:43.356372 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec56cce-7836-4cf4-a5bd-dfcabb478b3d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:43.356662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec56cce-7836-4cf4-a5bd-dfcabb478b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:43.356941 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec56cce-7836-4cf4-a5bd-dfcabb478b3d None admin] PATH_INFO: `/v3/groups/54b2eb315544408990a4259d8a76c132` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:43.358198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec56cce-7836-4cf4-a5bd-dfcabb478b3d None admin] RBAC: Authorizing `identity:get_group(group_id=54b2eb315544408990a4259d8a76c132)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:43.359726 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec56cce-7836-4cf4-a5bd-dfcabb478b3d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:43.361554 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 184/375] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jun 23 09:03:43 2022] GET /identity/v3/groups/54b2eb315544408990a4259d8a76c132 => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:43.363663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc41493-be1f-41bd-a5df-2128370bc88e None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:43.363663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc41493-be1f-41bd-a5df-2128370bc88e None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:43.363663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc41493-be1f-41bd-a5df-2128370bc88e None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:43.367280 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 185/376] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:43.368084 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc41493-be1f-41bd-a5df-2128370bc88e None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:43.376946 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 192/377] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:03:43.378597 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379306e-c0cc-4a5d-8c1b-2f66d77d6787 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:43.378894 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379306e-c0cc-4a5d-8c1b-2f66d77d6787 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:43.379198 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379306e-c0cc-4a5d-8c1b-2f66d77d6787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:43.399883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-8379306e-c0cc-4a5d-8c1b-2f66d77d6787 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:43.400999 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-8379306e-c0cc-4a5d-8c1b-2f66d77d6787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:43.483803 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:43.493343 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:43.499967 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc41493-be1f-41bd-a5df-2128370bc88e None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:43.507076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 193/378] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:43.507908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379306e-c0cc-4a5d-8c1b-2f66d77d6787 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:43.545352 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f47c46-baa3-4f2f-9926-67c1286fe333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:43.546014 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f47c46-baa3-4f2f-9926-67c1286fe333 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:43.546321 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f47c46-baa3-4f2f-9926-67c1286fe333 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:43.546579 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f47c46-baa3-4f2f-9926-67c1286fe333 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:43.547516 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f47c46-baa3-4f2f-9926-67c1286fe333 None admin] RBAC: Adding query filter params (group.id=54b2eb315544408990a4259d8a76c132, role.id=478b327fd22845cc98a2fafe821b1e85, scope.project.id=c0336e9e466e4bd4a28254482978f7b2) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:43.547833 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f47c46-baa3-4f2f-9926-67c1286fe333 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:43.549415 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f47c46-baa3-4f2f-9926-67c1286fe333 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:43.556368 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 186/379] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jun 23 09:03:43 2022] GET /identity/v3/role_assignments?group.id=54b2eb315544408990a4259d8a76c132&scope.project.id=c0336e9e466e4bd4a28254482978f7b2&role.id=478b327fd22845cc98a2fafe821b1e85 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:43.557252 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f47c46-baa3-4f2f-9926-67c1286fe333 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:43.664027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379306e-c0cc-4a5d-8c1b-2f66d77d6787 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:43.696509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f47c46-baa3-4f2f-9926-67c1286fe333 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:43.719935 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a402a-9516-4897-bd3a-46a3b1e5a5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:43.721209 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a402a-9516-4897-bd3a-46a3b1e5a5ec None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:43.721706 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a402a-9516-4897-bd3a-46a3b1e5a5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:43.722942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a402a-9516-4897-bd3a-46a3b1e5a5ec None admin] PATH_INFO: `/v3/projects/e5fa9b7cbdd44d0c8b5ffa0c15d81646` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:43.724680 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a402a-9516-4897-bd3a-46a3b1e5a5ec None admin] RBAC: Authorizing `identity:get_project(project_id=e5fa9b7cbdd44d0c8b5ffa0c15d81646)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:43.728053 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a402a-9516-4897-bd3a-46a3b1e5a5ec None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:43.731637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 194/380] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:43 2022] GET /identity/v3/projects/e5fa9b7cbdd44d0c8b5ffa0c15d81646 => generated 344 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:43.740029 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a402a-9516-4897-bd3a-46a3b1e5a5ec None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:43.777034 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82640d-76b0-401b-8e27-5f8486294076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:43.778433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82640d-76b0-401b-8e27-5f8486294076 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:43.778723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82640d-76b0-401b-8e27-5f8486294076 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:43.779033 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82640d-76b0-401b-8e27-5f8486294076 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:43.807991 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82640d-76b0-401b-8e27-5f8486294076 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:43.818411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-ea82640d-76b0-401b-8e27-5f8486294076 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:43.818411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:43.818411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:43.818411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:43.818411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:43.818411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:43.818411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:43.818411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:43.818411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:43.818411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:43.818411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:43.818411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:43.818411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:43.818411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:43.818411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:43.818411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:43.819469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:43.819469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:43.819469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:43.819469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:43.819469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:43.819469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:43.819469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:43.819469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:43.819469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:43.819469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:43.819469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:43.819469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:43.819469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:43.819469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:43.819469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:43.819469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:43.820625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:43.820625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:43.820625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:43.820625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:43.820625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:43.820625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:43.820625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:43.820625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:43.820625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:43.820625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:43.820625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:43.820625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:43.820625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 187/381] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:43.827137 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82640d-76b0-401b-8e27-5f8486294076 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:43.837685 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:43.849709 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:43.893471 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a402a-9516-4897-bd3a-46a3b1e5a5ec None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:43.966631 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff87bd-80ea-40da-99c1-e947f24f1af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:43.968189 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff87bd-80ea-40da-99c1-e947f24f1af4 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:43.968951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff87bd-80ea-40da-99c1-e947f24f1af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:43.969661 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff87bd-80ea-40da-99c1-e947f24f1af4 None admin] PATH_INFO: `/v3/groups/cc75182f2829402a8a6cb50cef5da0d6` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:43.978623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff87bd-80ea-40da-99c1-e947f24f1af4 None admin] RBAC: Authorizing `identity:get_group(group_id=cc75182f2829402a8a6cb50cef5da0d6)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:43.981213 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff87bd-80ea-40da-99c1-e947f24f1af4 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:43.982888 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82640d-76b0-401b-8e27-5f8486294076 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:43.987096 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 195/382] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jun 23 09:03:43 2022] GET /identity/v3/groups/cc75182f2829402a8a6cb50cef5da0d6 => generated 229 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:43.994561 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff87bd-80ea-40da-99c1-e947f24f1af4 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:44.031364 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4903c3a-d220-4c24-aebb-f2515fdc0d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:44.033871 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4903c3a-d220-4c24-aebb-f2515fdc0d4f None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:44.034421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4903c3a-d220-4c24-aebb-f2515fdc0d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:44.034939 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4903c3a-d220-4c24-aebb-f2515fdc0d4f None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:44.035982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4903c3a-d220-4c24-aebb-f2515fdc0d4f None admin] RBAC: Adding query filter params (name=member) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:44.036523 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4903c3a-d220-4c24-aebb-f2515fdc0d4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:44.038331 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4903c3a-d220-4c24-aebb-f2515fdc0d4f None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:44.048728 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 188/383] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:44.056318 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4903c3a-d220-4c24-aebb-f2515fdc0d4f None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:44.154041 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff87bd-80ea-40da-99c1-e947f24f1af4 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:44.169210 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:44.217116 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:44.227297 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e3d4b-4270-4ec9-b0ce-8be5ee77a84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:44.227972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e3d4b-4270-4ec9-b0ce-8be5ee77a84c None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:44.231436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e3d4b-4270-4ec9-b0ce-8be5ee77a84c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:44.231436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e3d4b-4270-4ec9-b0ce-8be5ee77a84c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:44.231436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e3d4b-4270-4ec9-b0ce-8be5ee77a84c None admin] RBAC: Adding query filter params (group.id=cc75182f2829402a8a6cb50cef5da0d6, role.id=9cd080b50c6f4ee8a54bf9ef1655bd83, scope.project.id=e5fa9b7cbdd44d0c8b5ffa0c15d81646) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:44.231436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e3d4b-4270-4ec9-b0ce-8be5ee77a84c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:44.231436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e3d4b-4270-4ec9-b0ce-8be5ee77a84c None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:44.237852 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 196/384] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jun 23 09:03:43 2022] GET /identity/v3/role_assignments?group.id=cc75182f2829402a8a6cb50cef5da0d6&scope.project.id=e5fa9b7cbdd44d0c8b5ffa0c15d81646&role.id=9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 260 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:44.244270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4903c3a-d220-4c24-aebb-f2515fdc0d4f None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:44.283539 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5c245-54cf-4e81-926b-36fd9a6c7ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:44.284418 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5c245-54cf-4e81-926b-36fd9a6c7ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:44.284700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5c245-54cf-4e81-926b-36fd9a6c7ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:44.284976 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5c245-54cf-4e81-926b-36fd9a6c7ee5 None admin] PATH_INFO: `/v3/groups/54b2eb315544408990a4259d8a76c132` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:44.286205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5c245-54cf-4e81-926b-36fd9a6c7ee5 None admin] RBAC: Authorizing `identity:get_group(group_id=54b2eb315544408990a4259d8a76c132)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:44.287709 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5c245-54cf-4e81-926b-36fd9a6c7ee5 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:44.289778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 189/385] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jun 23 09:03:44 2022] GET /identity/v3/groups/54b2eb315544408990a4259d8a76c132 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:44.295495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e3d4b-4270-4ec9-b0ce-8be5ee77a84c None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:44.407863 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e3d4b-4270-4ec9-b0ce-8be5ee77a84c None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:44.445867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca580120-840c-4ef8-99a1-ecddcbcc5917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:44.446751 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca580120-840c-4ef8-99a1-ecddcbcc5917 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:44.447070 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca580120-840c-4ef8-99a1-ecddcbcc5917 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:44.447351 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca580120-840c-4ef8-99a1-ecddcbcc5917 None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:44.448610 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca580120-840c-4ef8-99a1-ecddcbcc5917 None admin] RBAC: Authorizing `identity:get_project(project_id=79d2a60c71124efc90e0d0b445517d55)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:44.451512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca580120-840c-4ef8-99a1-ecddcbcc5917 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:44.454826 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 197/386] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:44 2022] GET /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:44.464071 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5c245-54cf-4e81-926b-36fd9a6c7ee5 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:44.593118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5c245-54cf-4e81-926b-36fd9a6c7ee5 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:44.597312 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ac75c-20dc-4f1e-8a58-0655066db2d0 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:44.597602 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ac75c-20dc-4f1e-8a58-0655066db2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:44.597880 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ac75c-20dc-4f1e-8a58-0655066db2d0 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:44.599580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 198/387] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:44.606654 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ccefe-6084-452d-b64e-39b95257153e None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:44.607291 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ccefe-6084-452d-b64e-39b95257153e None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:44.607631 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ccefe-6084-452d-b64e-39b95257153e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:44.631449 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-ca6ccefe-6084-452d-b64e-39b95257153e None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:44.633275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-ca6ccefe-6084-452d-b64e-39b95257153e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:44.679585 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165da347-b904-45a4-8954-a4fcb3c512f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:44.681447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165da347-b904-45a4-8954-a4fcb3c512f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:44.682976 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165da347-b904-45a4-8954-a4fcb3c512f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:44.688592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165da347-b904-45a4-8954-a4fcb3c512f1 None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55/groups/54b2eb315544408990a4259d8a76c132/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:44.691860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165da347-b904-45a4-8954-a4fcb3c512f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=79d2a60c71124efc90e0d0b445517d55, group_id=54b2eb315544408990a4259d8a76c132, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:44.693602 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165da347-b904-45a4-8954-a4fcb3c512f1 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:44.701631 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:44.719984 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:44.734142 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 199/388] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:03:44.758706 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ccefe-6084-452d-b64e-39b95257153e None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:44.783429 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 190/389] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Jun 23 09:03:44 2022] PUT /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55/groups/54b2eb315544408990a4259d8a76c132/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:03:44.903683 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ccefe-6084-452d-b64e-39b95257153e None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:44.950566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1dde2-5476-4a81-8f80-e0be546ba62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:44.951793 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1dde2-5476-4a81-8f80-e0be546ba62c None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:44.952264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1dde2-5476-4a81-8f80-e0be546ba62c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:44.952717 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1dde2-5476-4a81-8f80-e0be546ba62c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:44.970174 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1dde2-5476-4a81-8f80-e0be546ba62c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:44.979553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-91b1dde2-5476-4a81-8f80-e0be546ba62c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:44.979553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:44.979553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:44.979553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:44.979553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:44.979553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:44.979553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:44.979553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:44.979553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:44.979553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:44.979553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:44.979553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:44.979553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:44.979553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:44.979553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:44.979553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:44.980549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:44.980549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:44.980549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:44.980549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:44.980549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:44.980549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:44.980549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:44.980549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:44.980549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:44.980549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:44.980549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:44.980549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:44.980549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:44.980549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:44.980549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:44.980549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:44.981469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:44.981469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:44.981469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:44.981469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:44.981469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:44.981469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:44.981469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:44.981469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:44.981469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:44.981469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:44.981469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:44.981469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:44.983897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 200/390] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:03:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:44.997402 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165da347-b904-45a4-8954-a4fcb3c512f1 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:45.127186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165da347-b904-45a4-8954-a4fcb3c512f1 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:45.197248 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa25266-c511-448f-afed-b5704d8518ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:45.197973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa25266-c511-448f-afed-b5704d8518ac None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:45.198226 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa25266-c511-448f-afed-b5704d8518ac None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:45.198510 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa25266-c511-448f-afed-b5704d8518ac None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:45.199334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa25266-c511-448f-afed-b5704d8518ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:45.200191 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa25266-c511-448f-afed-b5704d8518ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:45.201592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa25266-c511-448f-afed-b5704d8518ac None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:45.214980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 191/391] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:03:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:45.220556 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1dde2-5476-4a81-8f80-e0be546ba62c None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:45.281870 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:45.337095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1dde2-5476-4a81-8f80-e0be546ba62c None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:45.378000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52ac3c-c65d-456e-868c-eb848cc28d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:45.378900 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52ac3c-c65d-456e-868c-eb848cc28d0c None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:45.379231 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52ac3c-c65d-456e-868c-eb848cc28d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:45.379599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52ac3c-c65d-456e-868c-eb848cc28d0c None admin] PATH_INFO: `/v3/groups/54b2eb315544408990a4259d8a76c132` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:45.380959 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52ac3c-c65d-456e-868c-eb848cc28d0c None admin] RBAC: Authorizing `identity:get_group(group_id=54b2eb315544408990a4259d8a76c132)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:45.382575 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52ac3c-c65d-456e-868c-eb848cc28d0c None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:45.384685 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 201/392] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jun 23 09:03:45 2022] GET /identity/v3/groups/54b2eb315544408990a4259d8a76c132 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:45.390385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa25266-c511-448f-afed-b5704d8518ac None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:45.463308 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:45.507425 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa25266-c511-448f-afed-b5704d8518ac None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:45.552794 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb2b08-6ec1-442b-9f0f-26ea52f4806c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:45.553733 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb2b08-6ec1-442b-9f0f-26ea52f4806c None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:45.554022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb2b08-6ec1-442b-9f0f-26ea52f4806c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:45.554303 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb2b08-6ec1-442b-9f0f-26ea52f4806c None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:45.555606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb2b08-6ec1-442b-9f0f-26ea52f4806c None admin] RBAC: Authorizing `identity:get_project(project_id=79d2a60c71124efc90e0d0b445517d55)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:45.557063 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb2b08-6ec1-442b-9f0f-26ea52f4806c None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:45.559618 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 192/393] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:45 2022] GET /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:45.565010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52ac3c-c65d-456e-868c-eb848cc28d0c None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:45.687102 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52ac3c-c65d-456e-868c-eb848cc28d0c None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:45.711016 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:45.725701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169260b-88ee-4c82-8cbb-b23a67dec352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:45.726608 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169260b-88ee-4c82-8cbb-b23a67dec352 None admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:45.726900 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169260b-88ee-4c82-8cbb-b23a67dec352 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:45.727201 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169260b-88ee-4c82-8cbb-b23a67dec352 None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55/groups/54b2eb315544408990a4259d8a76c132/roles/478b327fd22845cc98a2fafe821b1e85` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:45.733043 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169260b-88ee-4c82-8cbb-b23a67dec352 None admin] RBAC: Authorizing `identity:create_grant(project_id=79d2a60c71124efc90e0d0b445517d55, group_id=54b2eb315544408990a4259d8a76c132, role_id=478b327fd22845cc98a2fafe821b1e85)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:45.734492 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169260b-88ee-4c82-8cbb-b23a67dec352 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:45.807165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decc690-f9a8-433d-a298-b10c0008bd7c None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:45.807561 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decc690-f9a8-433d-a298-b10c0008bd7c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:45.807852 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decc690-f9a8-433d-a298-b10c0008bd7c None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:45.810381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 193/394] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:45.816476 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad606ff-bb95-4b11-a03d-f380f0c13713 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:45.816892 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad606ff-bb95-4b11-a03d-f380f0c13713 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:45.817309 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad606ff-bb95-4b11-a03d-f380f0c13713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:45.835481 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-aad606ff-bb95-4b11-a03d-f380f0c13713 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:45.836688 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-aad606ff-bb95-4b11-a03d-f380f0c13713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:45.862229 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 202/395] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Jun 23 09:03:45 2022] PUT /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55/groups/54b2eb315544408990a4259d8a76c132/roles/478b327fd22845cc98a2fafe821b1e85 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:03:45.936734 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 194/396] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:03:45.976803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169260b-88ee-4c82-8cbb-b23a67dec352 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:46.109589 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169260b-88ee-4c82-8cbb-b23a67dec352 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:46.155445 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651a4be-242a-4977-a087-22f9a855f981 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:46.155741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651a4be-242a-4977-a087-22f9a855f981 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:46.156021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651a4be-242a-4977-a087-22f9a855f981 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:46.157457 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 195/397] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:46.163158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756d020-c7b5-47b6-8c44-74d6d2f60063 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:46.163462 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756d020-c7b5-47b6-8c44-74d6d2f60063 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:46.163747 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756d020-c7b5-47b6-8c44-74d6d2f60063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:46.183559 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-8756d020-c7b5-47b6-8c44-74d6d2f60063 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:46.185338 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-8756d020-c7b5-47b6-8c44-74d6d2f60063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:46.190824 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d21e0c-1065-401e-a3f1-01ba614a7eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:46.191674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d21e0c-1065-401e-a3f1-01ba614a7eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:46.191962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d21e0c-1065-401e-a3f1-01ba614a7eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:46.192243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d21e0c-1065-401e-a3f1-01ba614a7eb5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:46.211849 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d21e0c-1065-401e-a3f1-01ba614a7eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:46.220452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-75d21e0c-1065-401e-a3f1-01ba614a7eb5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:46.220452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:46.220452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:46.220452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:46.220452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:46.220452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:46.220452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:46.220452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:46.220452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:46.220452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:46.220452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:46.220452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:46.220452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:46.220452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:46.220452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:46.220452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:46.221823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:46.221823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:46.221823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:46.221823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:46.221823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:46.221823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:46.221823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:46.221823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:46.221823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:46.221823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:46.221823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:46.221823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:46.221823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:46.221823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:46.221823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:46.221823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:46.222692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:46.222692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:46.222692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:46.222692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:46.222692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:46.222692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:46.222692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:46.222692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:46.222692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:46.222692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:46.222692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:46.222692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:46.222692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 203/398] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:46.227682 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d21e0c-1065-401e-a3f1-01ba614a7eb5 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:46.251890 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:46.272858 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 196/399] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:03:46.300574 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756d020-c7b5-47b6-8c44-74d6d2f60063 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:46.321322 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:46.425227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756d020-c7b5-47b6-8c44-74d6d2f60063 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:46.469595 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360107ef-7d27-451e-90bd-aac025f21613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:46.470769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360107ef-7d27-451e-90bd-aac025f21613 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:46.471322 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360107ef-7d27-451e-90bd-aac025f21613 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:46.471805 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360107ef-7d27-451e-90bd-aac025f21613 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:46.484717 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360107ef-7d27-451e-90bd-aac025f21613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:46.496781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-360107ef-7d27-451e-90bd-aac025f21613 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:46.496781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:46.496781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:46.496781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:46.496781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:46.496781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:46.496781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:46.496781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:46.496781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:46.496781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:46.496781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:46.496781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:46.496781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:46.496781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:46.496781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:46.496781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:46.497757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:46.497757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:46.497757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:46.497757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:46.497757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:46.497757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:46.497757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:46.497757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:46.497757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:46.497757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:46.497757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:46.497757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:46.497757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:46.497757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:46.497757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:46.497757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:46.498603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:46.498603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:46.498603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:46.498603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:46.498603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:46.498603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:46.498603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:46.498603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:46.498603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:46.498603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:46.498603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:46.498603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:46.501198 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d21e0c-1065-401e-a3f1-01ba614a7eb5 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:46.504513 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 197/400] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:03:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:46.509811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360107ef-7d27-451e-90bd-aac025f21613 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:46.576998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da5710-e1b8-46ae-9690-967d36873b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:46.577940 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da5710-e1b8-46ae-9690-967d36873b29 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:46.578403 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da5710-e1b8-46ae-9690-967d36873b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:46.578828 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da5710-e1b8-46ae-9690-967d36873b29 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:46.579756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da5710-e1b8-46ae-9690-967d36873b29 None admin] RBAC: Adding query filter params (name=member) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:46.580216 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da5710-e1b8-46ae-9690-967d36873b29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:46.581809 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da5710-e1b8-46ae-9690-967d36873b29 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:46.595752 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:46.600437 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 204/401] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:46.606572 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da5710-e1b8-46ae-9690-967d36873b29 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:46.701314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360107ef-7d27-451e-90bd-aac025f21613 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:46.773246 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da5710-e1b8-46ae-9690-967d36873b29 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:46.775693 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:46.778700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7724373-e74a-4026-b3c8-94831bfa14f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:46.779436 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7724373-e74a-4026-b3c8-94831bfa14f5 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:46.779723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7724373-e74a-4026-b3c8-94831bfa14f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:46.780002 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7724373-e74a-4026-b3c8-94831bfa14f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:46.780782 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7724373-e74a-4026-b3c8-94831bfa14f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:46.781088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7724373-e74a-4026-b3c8-94831bfa14f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:46.782520 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7724373-e74a-4026-b3c8-94831bfa14f5 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:46.792023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 198/402] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:03:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:46.798620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7724373-e74a-4026-b3c8-94831bfa14f5 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:46.879642 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c344d2c-83da-4b32-86d5-6eb60b54862d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:46.880533 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c344d2c-83da-4b32-86d5-6eb60b54862d None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:46.880822 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c344d2c-83da-4b32-86d5-6eb60b54862d None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:46.881098 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c344d2c-83da-4b32-86d5-6eb60b54862d None admin] PATH_INFO: `/v3/groups/54b2eb315544408990a4259d8a76c132` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:46.882349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c344d2c-83da-4b32-86d5-6eb60b54862d None admin] RBAC: Authorizing `identity:get_group(group_id=54b2eb315544408990a4259d8a76c132)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:46.883853 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c344d2c-83da-4b32-86d5-6eb60b54862d None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:46.885829 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 205/403] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jun 23 09:03:46 2022] GET /identity/v3/groups/54b2eb315544408990a4259d8a76c132 => generated 226 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:46.891295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c344d2c-83da-4b32-86d5-6eb60b54862d None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:46.918089 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7724373-e74a-4026-b3c8-94831bfa14f5 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:46.928479 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:46.961071 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4d36c-c943-4851-baf6-ce66fbaf0630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:46.962262 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4d36c-c943-4851-baf6-ce66fbaf0630 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:46.962697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4d36c-c943-4851-baf6-ce66fbaf0630 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:46.963148 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4d36c-c943-4851-baf6-ce66fbaf0630 None admin] PATH_INFO: `/v3/groups/54b2eb315544408990a4259d8a76c132` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:46.964877 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4d36c-c943-4851-baf6-ce66fbaf0630 None admin] RBAC: Authorizing `identity:get_group(group_id=54b2eb315544408990a4259d8a76c132)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:46.966756 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4d36c-c943-4851-baf6-ce66fbaf0630 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:46.973186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 199/404] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jun 23 09:03:46 2022] GET /identity/v3/groups/54b2eb315544408990a4259d8a76c132 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:46.978677 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4d36c-c943-4851-baf6-ce66fbaf0630 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:47.112608 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c344d2c-83da-4b32-86d5-6eb60b54862d None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:47.156701 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4d36c-c943-4851-baf6-ce66fbaf0630 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:47.176830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c055fc-8730-400f-8f35-aa794f92e762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:47.177706 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c055fc-8730-400f-8f35-aa794f92e762 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:47.177998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c055fc-8730-400f-8f35-aa794f92e762 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:47.178280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c055fc-8730-400f-8f35-aa794f92e762 None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:47.179532 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c055fc-8730-400f-8f35-aa794f92e762 None admin] RBAC: Authorizing `identity:get_project(project_id=c0336e9e466e4bd4a28254482978f7b2)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:47.180988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c055fc-8730-400f-8f35-aa794f92e762 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:47.184389 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 206/405] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:46 2022] GET /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2 => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:47.195930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c055fc-8730-400f-8f35-aa794f92e762 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:47.214844 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d1d4d-bfbf-4a5c-b907-b437f19a482f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:47.215786 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d1d4d-bfbf-4a5c-b907-b437f19a482f None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:47.216073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d1d4d-bfbf-4a5c-b907-b437f19a482f None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:47.216352 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d1d4d-bfbf-4a5c-b907-b437f19a482f None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:47.218769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d1d4d-bfbf-4a5c-b907-b437f19a482f None admin] RBAC: Authorizing `identity:get_project(project_id=c0336e9e466e4bd4a28254482978f7b2)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:47.220266 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d1d4d-bfbf-4a5c-b907-b437f19a482f None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:47.222290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 200/406] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:46 2022] GET /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:47.227859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d1d4d-bfbf-4a5c-b907-b437f19a482f None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:47.282123 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:47.283334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:47.346001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c055fc-8730-400f-8f35-aa794f92e762 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:47.346749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d1d4d-bfbf-4a5c-b907-b437f19a482f None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:47.383605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318940c-5385-4ae0-933c-490e2a225c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:47.384513 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318940c-5385-4ae0-933c-490e2a225c81 None admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:47.384919 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318940c-5385-4ae0-933c-490e2a225c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:47.385202 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318940c-5385-4ae0-933c-490e2a225c81 None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2/groups/54b2eb315544408990a4259d8a76c132/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:47.387905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318940c-5385-4ae0-933c-490e2a225c81 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0336e9e466e4bd4a28254482978f7b2, group_id=54b2eb315544408990a4259d8a76c132, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:47.389601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318940c-5385-4ae0-933c-490e2a225c81 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:47.390737 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91566c-6208-40b6-b3dc-eab76d833459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:47.391851 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91566c-6208-40b6-b3dc-eab76d833459 None admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:47.391851 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91566c-6208-40b6-b3dc-eab76d833459 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:47.391851 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91566c-6208-40b6-b3dc-eab76d833459 None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2/groups/54b2eb315544408990a4259d8a76c132/roles/478b327fd22845cc98a2fafe821b1e85` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:47.394928 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c91566c-6208-40b6-b3dc-eab76d833459 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0336e9e466e4bd4a28254482978f7b2, group_id=54b2eb315544408990a4259d8a76c132, role_id=478b327fd22845cc98a2fafe821b1e85)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:47.396620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c91566c-6208-40b6-b3dc-eab76d833459 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:47.509619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 207/407] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Jun 23 09:03:47 2022] PUT /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2/groups/54b2eb315544408990a4259d8a76c132/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:03:47.510864 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965cf407-feab-40a0-9c61-4671e75e7622 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:47.511189 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965cf407-feab-40a0-9c61-4671e75e7622 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:47.511469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965cf407-feab-40a0-9c61-4671e75e7622 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:47.513454 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 208/408] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:47.519265 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c7118-8f49-469b-8e2d-18f2c288ad99 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:47.527099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c7118-8f49-469b-8e2d-18f2c288ad99 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:47.527099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c7118-8f49-469b-8e2d-18f2c288ad99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:47.539214 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 201/409] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Jun 23 09:03:47 2022] PUT /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2/groups/54b2eb315544408990a4259d8a76c132/roles/478b327fd22845cc98a2fafe821b1e85 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:03:47.552795 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-6f7c7118-8f49-469b-8e2d-18f2c288ad99 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:47.554027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-6f7c7118-8f49-469b-8e2d-18f2c288ad99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:47.630578 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 209/410] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:03:47.657848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91566c-6208-40b6-b3dc-eab76d833459 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:47.717087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a7f24-4f47-491f-a6c5-ebc1cbf74605 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:47.717402 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a7f24-4f47-491f-a6c5-ebc1cbf74605 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:47.717656 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a7f24-4f47-491f-a6c5-ebc1cbf74605 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:47.719385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 210/411] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:47.724944 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad63e3-13e1-4a97-85d2-9913bc35605f None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:47.725223 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad63e3-13e1-4a97-85d2-9913bc35605f None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:47.725516 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad63e3-13e1-4a97-85d2-9913bc35605f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:47.760589 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-bfad63e3-13e1-4a97-85d2-9913bc35605f None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:47.761815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-bfad63e3-13e1-4a97-85d2-9913bc35605f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:47.804461 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:47.820714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91566c-6208-40b6-b3dc-eab76d833459 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:47.825308 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:47.866932 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf62afa-a136-4834-b461-f3467ba0b95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:47.867472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 211/412] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:47.870951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf62afa-a136-4834-b461-f3467ba0b95c None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:47.870951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf62afa-a136-4834-b461-f3467ba0b95c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:47.870951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf62afa-a136-4834-b461-f3467ba0b95c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:47.883878 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf62afa-a136-4834-b461-f3467ba0b95c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:47.892301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-7cf62afa-a136-4834-b461-f3467ba0b95c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:03:47.892301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:47.892301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:47.892301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:47.892301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:47.892301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:47.892301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:47.892301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:47.892301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:47.892301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:47.892301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:47.892301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:47.892301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:47.892301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:47.892301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:47.892301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:47.893468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:47.893468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:47.893468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:47.893468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:47.893468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:47.893468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:47.893468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:47.893468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:47.893468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:47.893468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:47.893468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:47.893468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:47.893468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:47.893468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:47.893468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:47.893468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:47.894562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:47.894562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:47.894562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:47.894562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:47.894562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:47.894562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:47.894562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:47.894562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:47.894562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:47.894562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:47.894562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:03:47.894562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:47.897120 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad63e3-13e1-4a97-85d2-9913bc35605f None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:47.902465 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 202/413] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:03:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:47.905321 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf62afa-a136-4834-b461-f3467ba0b95c None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:48.073163 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf62afa-a136-4834-b461-f3467ba0b95c None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:48.110253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62c5b7-f15b-45cd-b977-3a5407aa7f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:48.111092 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62c5b7-f15b-45cd-b977-3a5407aa7f0d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:48.111441 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62c5b7-f15b-45cd-b977-3a5407aa7f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:48.111784 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62c5b7-f15b-45cd-b977-3a5407aa7f0d None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:48.112610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62c5b7-f15b-45cd-b977-3a5407aa7f0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:48.112987 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62c5b7-f15b-45cd-b977-3a5407aa7f0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:48.113970 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad63e3-13e1-4a97-85d2-9913bc35605f None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:48.114517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62c5b7-f15b-45cd-b977-3a5407aa7f0d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:48.124023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 203/414] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:48.129535 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62c5b7-f15b-45cd-b977-3a5407aa7f0d None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:48.175219 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12319f0c-2a78-4013-8c96-f427f1f9dddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:48.176043 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12319f0c-2a78-4013-8c96-f427f1f9dddd None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:48.176329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12319f0c-2a78-4013-8c96-f427f1f9dddd None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:48.176607 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12319f0c-2a78-4013-8c96-f427f1f9dddd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:48.190824 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12319f0c-2a78-4013-8c96-f427f1f9dddd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:48.202500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-12319f0c-2a78-4013-8c96-f427f1f9dddd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:48.202500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:48.202500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:48.202500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:48.202500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:48.202500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:48.202500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:48.202500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:48.202500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:48.202500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:48.202500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:48.202500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:48.202500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:48.202500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:48.202500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:48.202500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:48.203512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:48.203512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:48.203512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:48.203512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:48.203512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:48.203512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:48.203512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:48.203512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:48.203512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:48.203512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:48.203512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:48.203512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:48.203512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:48.203512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:48.203512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:48.203512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:48.204758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:48.204758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:48.204758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:48.204758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:48.204758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:48.204758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:48.204758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:48.204758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:48.204758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:48.204758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:48.204758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:48.204758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:48.204758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 212/415] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:48.211430 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12319f0c-2a78-4013-8c96-f427f1f9dddd None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:48.214006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:48.252304 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:48.294939 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62c5b7-f15b-45cd-b977-3a5407aa7f0d None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:48.340190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632b057-8c3c-4455-902f-088c030dc46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:48.340995 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632b057-8c3c-4455-902f-088c030dc46e None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:48.341294 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632b057-8c3c-4455-902f-088c030dc46e None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:48.341577 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632b057-8c3c-4455-902f-088c030dc46e None admin] PATH_INFO: `/v3/groups/cc75182f2829402a8a6cb50cef5da0d6` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:48.343465 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632b057-8c3c-4455-902f-088c030dc46e None admin] RBAC: Authorizing `identity:get_group(group_id=cc75182f2829402a8a6cb50cef5da0d6)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:48.344943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632b057-8c3c-4455-902f-088c030dc46e None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:48.346798 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 204/416] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jun 23 09:03:48 2022] GET /identity/v3/groups/cc75182f2829402a8a6cb50cef5da0d6 => generated 229 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:48.357271 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632b057-8c3c-4455-902f-088c030dc46e None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:48.419231 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12319f0c-2a78-4013-8c96-f427f1f9dddd None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:48.454642 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:48.457101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde44ba-5461-45b1-a294-a674b86b72ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:48.458286 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde44ba-5461-45b1-a294-a674b86b72ac None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:48.458799 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde44ba-5461-45b1-a294-a674b86b72ac None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:48.459310 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde44ba-5461-45b1-a294-a674b86b72ac None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:48.460259 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde44ba-5461-45b1-a294-a674b86b72ac None admin] RBAC: Adding query filter params (name=member) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:48.460774 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde44ba-5461-45b1-a294-a674b86b72ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:48.461503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632b057-8c3c-4455-902f-088c030dc46e None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:48.462582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde44ba-5461-45b1-a294-a674b86b72ac None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:48.472696 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 213/417] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:48.474386 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f742d-a26c-4a55-b0c2-2337a8a66503 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:48.474897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f742d-a26c-4a55-b0c2-2337a8a66503 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:48.475395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f742d-a26c-4a55-b0c2-2337a8a66503 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:48.476906 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 214/418] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:48.478170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f742d-a26c-4a55-b0c2-2337a8a66503 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:48.502778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08c4c3-0f62-4603-9215-08af16af1f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:48.503700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08c4c3-0f62-4603-9215-08af16af1f0c None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:48.503991 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08c4c3-0f62-4603-9215-08af16af1f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:48.504273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08c4c3-0f62-4603-9215-08af16af1f0c None admin] PATH_INFO: `/v3/projects/e5fa9b7cbdd44d0c8b5ffa0c15d81646` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:48.505474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08c4c3-0f62-4603-9215-08af16af1f0c None admin] RBAC: Authorizing `identity:get_project(project_id=e5fa9b7cbdd44d0c8b5ffa0c15d81646)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:48.506946 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08c4c3-0f62-4603-9215-08af16af1f0c None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:48.508980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 205/419] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:48 2022] GET /identity/v3/projects/e5fa9b7cbdd44d0c8b5ffa0c15d81646 => generated 344 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:48.510134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a9ce7-21af-4952-b321-7eddf206653a None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:48.510425 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a9ce7-21af-4952-b321-7eddf206653a None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:48.510709 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a9ce7-21af-4952-b321-7eddf206653a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:48.532461 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-e89a9ce7-21af-4952-b321-7eddf206653a None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:48.533668 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-e89a9ce7-21af-4952-b321-7eddf206653a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:48.572497 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:48.595802 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f742d-a26c-4a55-b0c2-2337a8a66503 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:48.603067 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 206/420] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:48.603966 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a9ce7-21af-4952-b321-7eddf206653a None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:48.632584 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302faff-3eec-4acc-85fa-9343aa4336c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:48.633696 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302faff-3eec-4acc-85fa-9343aa4336c9 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:48.634377 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302faff-3eec-4acc-85fa-9343aa4336c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:48.634963 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302faff-3eec-4acc-85fa-9343aa4336c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:48.698559 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:48.710381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a9ce7-21af-4952-b321-7eddf206653a None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:48.719351 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302faff-3eec-4acc-85fa-9343aa4336c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:48.721205 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302faff-3eec-4acc-85fa-9343aa4336c9 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:48.752263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170d17b-e7e9-4a5e-bfd3-db515f0fde0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:48.753549 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170d17b-e7e9-4a5e-bfd3-db515f0fde0c None admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:48.754078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170d17b-e7e9-4a5e-bfd3-db515f0fde0c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:48.755109 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170d17b-e7e9-4a5e-bfd3-db515f0fde0c None admin] PATH_INFO: `/v3/projects/e5fa9b7cbdd44d0c8b5ffa0c15d81646/groups/cc75182f2829402a8a6cb50cef5da0d6/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:48.757832 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170d17b-e7e9-4a5e-bfd3-db515f0fde0c None admin] RBAC: Authorizing `identity:create_grant(project_id=e5fa9b7cbdd44d0c8b5ffa0c15d81646, group_id=cc75182f2829402a8a6cb50cef5da0d6, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:48.760202 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170d17b-e7e9-4a5e-bfd3-db515f0fde0c None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:48.818241 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:48.835217 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 207/421] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Jun 23 09:03:48 2022] PUT /identity/v3/projects/e5fa9b7cbdd44d0c8b5ffa0c15d81646/groups/cc75182f2829402a8a6cb50cef5da0d6/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:03:48.836704 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170d17b-e7e9-4a5e-bfd3-db515f0fde0c None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:48.842973 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 215/422] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:03:48.843888 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302faff-3eec-4acc-85fa-9343aa4336c9 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:48.976755 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170d17b-e7e9-4a5e-bfd3-db515f0fde0c None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:49.020279 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ea4a2-1508-4c47-b333-7ba7b35ab714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:49.025149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ea4a2-1508-4c47-b333-7ba7b35ab714 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:49.026774 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ea4a2-1508-4c47-b333-7ba7b35ab714 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:49.027346 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ea4a2-1508-4c47-b333-7ba7b35ab714 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:49.040410 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ea4a2-1508-4c47-b333-7ba7b35ab714 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:49.041009 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302faff-3eec-4acc-85fa-9343aa4336c9 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:49.051699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-db6ea4a2-1508-4c47-b333-7ba7b35ab714 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:49.051699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:49.051699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:49.051699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:49.051699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:49.051699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:49.051699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:49.051699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:49.051699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:49.051699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:49.051699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:49.051699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:49.051699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:49.051699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:49.051699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:49.051699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:49.052727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:49.052727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:49.052727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:49.052727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:49.052727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:49.052727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:49.052727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:49.052727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:49.052727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:49.052727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:49.052727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:49.052727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:49.052727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:49.052727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:49.052727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:49.052727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:49.053573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:49.053573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:49.053573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:49.053573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:49.053573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:49.053573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:49.053573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:49.053573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:49.053573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:49.053573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:49.053573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:49.053573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:49.056930 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 208/423] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:03:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:49.063638 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ea4a2-1508-4c47-b333-7ba7b35ab714 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:49.086540 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:49.091296 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb20150-550c-43ea-b406-115b36c886b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:49.092310 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb20150-550c-43ea-b406-115b36c886b2 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:49.092739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb20150-550c-43ea-b406-115b36c886b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:49.093068 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb20150-550c-43ea-b406-115b36c886b2 None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:49.095001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb20150-550c-43ea-b406-115b36c886b2 None admin] RBAC: Authorizing `identity:get_project(project_id=79d2a60c71124efc90e0d0b445517d55)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:49.096625 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb20150-550c-43ea-b406-115b36c886b2 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:49.098942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 216/424] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:48 2022] GET /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:49.105703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb20150-550c-43ea-b406-115b36c886b2 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:49.182369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:49.202663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ea4a2-1508-4c47-b333-7ba7b35ab714 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:49.239007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a606f8e-698e-4eb6-a634-7797cd05d818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:49.239931 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a606f8e-698e-4eb6-a634-7797cd05d818 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:49.240340 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a606f8e-698e-4eb6-a634-7797cd05d818 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:49.240721 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a606f8e-698e-4eb6-a634-7797cd05d818 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:49.241586 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a606f8e-698e-4eb6-a634-7797cd05d818 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:49.241998 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a606f8e-698e-4eb6-a634-7797cd05d818 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:49.243558 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a606f8e-698e-4eb6-a634-7797cd05d818 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:49.247411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb20150-550c-43ea-b406-115b36c886b2 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:49.255369 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 209/425] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:03:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:49.262857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a606f8e-698e-4eb6-a634-7797cd05d818 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:49.309564 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35292f2-c461-4549-aaf8-c22b2381ea51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:49.312173 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35292f2-c461-4549-aaf8-c22b2381ea51 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:49.312465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35292f2-c461-4549-aaf8-c22b2381ea51 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:49.312744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35292f2-c461-4549-aaf8-c22b2381ea51 None admin] PATH_INFO: `/v3/groups/54b2eb315544408990a4259d8a76c132` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:49.316510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35292f2-c461-4549-aaf8-c22b2381ea51 None admin] RBAC: Authorizing `identity:get_group(group_id=54b2eb315544408990a4259d8a76c132)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:49.317959 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35292f2-c461-4549-aaf8-c22b2381ea51 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:49.322100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 217/426] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jun 23 09:03:49 2022] GET /identity/v3/groups/54b2eb315544408990a4259d8a76c132 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:49.331550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35292f2-c461-4549-aaf8-c22b2381ea51 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:49.405375 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:49.453317 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a606f8e-698e-4eb6-a634-7797cd05d818 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:49.481449 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35292f2-c461-4549-aaf8-c22b2381ea51 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:49.500769 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:49.504554 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a5537-bd9a-4a2e-861c-f5594e5455ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:49.505231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a5537-bd9a-4a2e-861c-f5594e5455ff None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:49.505521 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a5537-bd9a-4a2e-861c-f5594e5455ff None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:49.505800 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a5537-bd9a-4a2e-861c-f5594e5455ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:49.531033 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151720a-3000-47ad-8475-775393f62119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:49.532642 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151720a-3000-47ad-8475-775393f62119 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:49.532957 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151720a-3000-47ad-8475-775393f62119 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:49.533236 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151720a-3000-47ad-8475-775393f62119 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:49.535758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151720a-3000-47ad-8475-775393f62119 None admin] RBAC: Adding query filter params (group.id=54b2eb315544408990a4259d8a76c132, role.id=53a3d65343394e7caafcc33f12cf138b, scope.project.id=79d2a60c71124efc90e0d0b445517d55) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:49.536078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151720a-3000-47ad-8475-775393f62119 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:49.537537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151720a-3000-47ad-8475-775393f62119 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:49.553599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 218/427] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jun 23 09:03:49 2022] GET /identity/v3/role_assignments?group.id=54b2eb315544408990a4259d8a76c132&scope.project.id=79d2a60c71124efc90e0d0b445517d55&role.id=53a3d65343394e7caafcc33f12cf138b => generated 611 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:49.617384 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a5537-bd9a-4a2e-861c-f5594e5455ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:49.618895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a5537-bd9a-4a2e-861c-f5594e5455ff None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:49.730048 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:49.757647 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 210/428] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:03:49.763479 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151720a-3000-47ad-8475-775393f62119 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:49.877754 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151720a-3000-47ad-8475-775393f62119 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:49.916361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b1724-7fc2-4d86-a070-ec44d6782270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:49.917232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b1724-7fc2-4d86-a070-ec44d6782270 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:49.917513 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b1724-7fc2-4d86-a070-ec44d6782270 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:49.917787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b1724-7fc2-4d86-a070-ec44d6782270 None admin] PATH_INFO: `/v3/projects/79d2a60c71124efc90e0d0b445517d55` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:49.919054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b1724-7fc2-4d86-a070-ec44d6782270 None admin] RBAC: Authorizing `identity:get_project(project_id=79d2a60c71124efc90e0d0b445517d55)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:49.920500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b1724-7fc2-4d86-a070-ec44d6782270 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:49.922204 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 219/429] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:49 2022] GET /identity/v3/projects/79d2a60c71124efc90e0d0b445517d55 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:49.927893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a5537-bd9a-4a2e-861c-f5594e5455ff None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:50.031605 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a5537-bd9a-4a2e-861c-f5594e5455ff None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:50.066928 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d1fc7-d5d8-4d08-bc67-3af9ba3ef380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:50.067818 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d1fc7-d5d8-4d08-bc67-3af9ba3ef380 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:50.068101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d1fc7-d5d8-4d08-bc67-3af9ba3ef380 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:50.069128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d1fc7-d5d8-4d08-bc67-3af9ba3ef380 None admin] PATH_INFO: `/v3/groups/54b2eb315544408990a4259d8a76c132` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:50.069601 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d1fc7-d5d8-4d08-bc67-3af9ba3ef380 None admin] RBAC: Authorizing `identity:get_group(group_id=54b2eb315544408990a4259d8a76c132)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:50.071070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d1fc7-d5d8-4d08-bc67-3af9ba3ef380 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:50.073033 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 211/430] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jun 23 09:03:49 2022] GET /identity/v3/groups/54b2eb315544408990a4259d8a76c132 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:50.080052 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b1724-7fc2-4d86-a070-ec44d6782270 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:50.102806 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bd98d-3939-4450-a01c-0bf718bda3f8 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:50.109538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bd98d-3939-4450-a01c-0bf718bda3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:50.109538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bd98d-3939-4450-a01c-0bf718bda3f8 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:50.109538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 212/431] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:03:50.116195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb68e2-4bbe-43b0-9ad7-009fd78cea35 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:50.117038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb68e2-4bbe-43b0-9ad7-009fd78cea35 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:50.117424 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb68e2-4bbe-43b0-9ad7-009fd78cea35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:50.138642 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-69cb68e2-4bbe-43b0-9ad7-009fd78cea35 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:50.140492 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-69cb68e2-4bbe-43b0-9ad7-009fd78cea35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:50.142453 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:50.200350 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:50.204478 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b1724-7fc2-4d86-a070-ec44d6782270 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:50.244158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 213/432] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:03:50.250936 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0436585-7e51-4bf5-84d5-b0fc7e11ea17 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:50.250936 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0436585-7e51-4bf5-84d5-b0fc7e11ea17 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:50.250936 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0436585-7e51-4bf5-84d5-b0fc7e11ea17 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:50.250936 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 214/433] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:50.253247 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f90e3-2bea-473e-bfe9-b2417e16e060 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:50.253539 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f90e3-2bea-473e-bfe9-b2417e16e060 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:50.253816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f90e3-2bea-473e-bfe9-b2417e16e060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:50.256323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf77f81-ebe1-484f-ba01-3b017bf92654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:50.257246 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf77f81-ebe1-484f-ba01-3b017bf92654 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:50.257736 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf77f81-ebe1-484f-ba01-3b017bf92654 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:50.258216 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf77f81-ebe1-484f-ba01-3b017bf92654 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:50.259421 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf77f81-ebe1-484f-ba01-3b017bf92654 None admin] RBAC: Adding query filter params (group.id=54b2eb315544408990a4259d8a76c132, role.id=478b327fd22845cc98a2fafe821b1e85, scope.project.id=79d2a60c71124efc90e0d0b445517d55) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:50.260005 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf77f81-ebe1-484f-ba01-3b017bf92654 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:50.261455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf77f81-ebe1-484f-ba01-3b017bf92654 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:50.269139 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 220/434] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jun 23 09:03:50 2022] GET /identity/v3/role_assignments?group.id=54b2eb315544408990a4259d8a76c132&scope.project.id=79d2a60c71124efc90e0d0b445517d55&role.id=478b327fd22845cc98a2fafe821b1e85 => generated 611 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:50.272860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-0e8f90e3-2bea-473e-bfe9-b2417e16e060 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:50.273020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf77f81-ebe1-484f-ba01-3b017bf92654 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:50.274373 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-0e8f90e3-2bea-473e-bfe9-b2417e16e060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:50.379218 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 215/435] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:50.411424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf77f81-ebe1-484f-ba01-3b017bf92654 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:50.412135 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f90e3-2bea-473e-bfe9-b2417e16e060 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:50.430376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:50.457071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec74dcf-67db-4875-8bda-2e6077819f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:50.457886 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec74dcf-67db-4875-8bda-2e6077819f14 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:50.458544 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec74dcf-67db-4875-8bda-2e6077819f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:50.458829 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec74dcf-67db-4875-8bda-2e6077819f14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:50.471060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec74dcf-67db-4875-8bda-2e6077819f14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:50.479784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-3ec74dcf-67db-4875-8bda-2e6077819f14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:50.479784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:50.479784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:50.479784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:50.479784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:50.479784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:50.479784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:50.479784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:50.479784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:50.479784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:50.479784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:50.479784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:50.479784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:50.479784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:50.479784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:50.479784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:50.481338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:50.481338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:50.481338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:50.481338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:50.481338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:50.481338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:50.481338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:50.481338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:50.481338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:50.481338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:50.481338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:50.481338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:50.481338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:50.481338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:50.481338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:50.481338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:50.482616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:50.482616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:50.482616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:50.482616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:50.482616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:50.482616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:50.482616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:50.482616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:50.482616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:50.482616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:50.482616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 09:03:50.482616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:50.482616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 221/436] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:50.488976 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec74dcf-67db-4875-8bda-2e6077819f14 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:50.547675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f90e3-2bea-473e-bfe9-b2417e16e060 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:50.557119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:50.595527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417ee22-15e9-427b-baae-80c1998d99ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:50.597091 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417ee22-15e9-427b-baae-80c1998d99ee None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:50.597091 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417ee22-15e9-427b-baae-80c1998d99ee None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:50.597275 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417ee22-15e9-427b-baae-80c1998d99ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:50.612978 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417ee22-15e9-427b-baae-80c1998d99ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:50.617428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec74dcf-67db-4875-8bda-2e6077819f14 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:50.623718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-1417ee22-15e9-427b-baae-80c1998d99ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:50.623718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:50.623718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:50.623718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:50.623718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:50.623718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:50.623718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:50.623718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:50.623718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:50.623718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:50.623718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:50.623718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:50.623718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:50.623718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:50.623718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:50.623718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:50.624682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:50.624682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:50.624682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:50.624682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:50.624682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:50.624682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:50.624682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:50.624682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:50.624682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:50.624682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:50.624682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:50.624682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:50.624682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:50.624682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:50.624682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:50.624682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:50.625553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:50.625553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:50.625553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:50.625553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:50.625553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:50.625553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:50.625553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:50.625553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:50.625553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:50.625553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:50.625553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:50.625553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:50.625553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 216/437] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:03:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:50.630260 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417ee22-15e9-427b-baae-80c1998d99ee None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:50.667941 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befcd8e-894a-40d4-9dc2-627e14c729b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:50.668693 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befcd8e-894a-40d4-9dc2-627e14c729b2 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:50.668983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befcd8e-894a-40d4-9dc2-627e14c729b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:50.669290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befcd8e-894a-40d4-9dc2-627e14c729b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:50.670164 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befcd8e-894a-40d4-9dc2-627e14c729b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:50.670480 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befcd8e-894a-40d4-9dc2-627e14c729b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:50.673596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befcd8e-894a-40d4-9dc2-627e14c729b2 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:50.685465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 222/438] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:03:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:50.691179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befcd8e-894a-40d4-9dc2-627e14c729b2 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:50.737485 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:50.773987 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417ee22-15e9-427b-baae-80c1998d99ee None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:50.812006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5674139-cd22-489d-a0b5-e6a1ffc06edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:50.812683 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5674139-cd22-489d-a0b5-e6a1ffc06edb None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:50.812972 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5674139-cd22-489d-a0b5-e6a1ffc06edb None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:50.813251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5674139-cd22-489d-a0b5-e6a1ffc06edb None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:50.814005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5674139-cd22-489d-a0b5-e6a1ffc06edb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:50.814314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5674139-cd22-489d-a0b5-e6a1ffc06edb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:50.815831 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5674139-cd22-489d-a0b5-e6a1ffc06edb None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:50.819996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befcd8e-894a-40d4-9dc2-627e14c729b2 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:50.824036 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 217/439] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:03:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:50.829783 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5674139-cd22-489d-a0b5-e6a1ffc06edb None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:50.857310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:50.859088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba14cb7-55b0-4715-a3fb-3165571fd0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:50.859800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba14cb7-55b0-4715-a3fb-3165571fd0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:50.860085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba14cb7-55b0-4715-a3fb-3165571fd0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:50.860366 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba14cb7-55b0-4715-a3fb-3165571fd0a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:50.937168 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5674139-cd22-489d-a0b5-e6a1ffc06edb None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:50.941004 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba14cb7-55b0-4715-a3fb-3165571fd0a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:50.942482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba14cb7-55b0-4715-a3fb-3165571fd0a5 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:50.973303 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2327b50-167c-4a73-84fc-46e1f6d16b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:50.973939 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2327b50-167c-4a73-84fc-46e1f6d16b38 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:50.974229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2327b50-167c-4a73-84fc-46e1f6d16b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:50.974509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2327b50-167c-4a73-84fc-46e1f6d16b38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:50.979744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:51.048810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 223/440] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 09:03:51.050609 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2327b50-167c-4a73-84fc-46e1f6d16b38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:51.052183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2327b50-167c-4a73-84fc-46e1f6d16b38 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:51.054055 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba14cb7-55b0-4715-a3fb-3165571fd0a5 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:51.079011 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:51.164259 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba14cb7-55b0-4715-a3fb-3165571fd0a5 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:51.165715 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 218/441] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 09:03:51.171304 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2327b50-167c-4a73-84fc-46e1f6d16b38 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:51.206040 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba9407-6f4a-45c4-b4f4-1e797bebb116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:51.206810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba9407-6f4a-45c4-b4f4-1e797bebb116 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:51.207128 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba9407-6f4a-45c4-b4f4-1e797bebb116 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:51.207410 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba9407-6f4a-45c4-b4f4-1e797bebb116 None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:51.209093 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba9407-6f4a-45c4-b4f4-1e797bebb116 None admin] RBAC: Authorizing `identity:get_project(project_id=c0336e9e466e4bd4a28254482978f7b2)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:51.210534 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba9407-6f4a-45c4-b4f4-1e797bebb116 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:51.212410 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 224/442] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:51 2022] GET /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:51.217947 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba9407-6f4a-45c4-b4f4-1e797bebb116 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:51.220780 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:51.279191 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2327b50-167c-4a73-84fc-46e1f6d16b38 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:51.303086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:51.314148 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ceba4-d295-4f53-9119-1022b58d43c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:51.315034 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ceba4-d295-4f53-9119-1022b58d43c7 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:51.315318 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ceba4-d295-4f53-9119-1022b58d43c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:51.315593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ceba4-d295-4f53-9119-1022b58d43c7 None admin] PATH_INFO: `/v3/projects/c0336e9e466e4bd4a28254482978f7b2` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:51.316809 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ceba4-d295-4f53-9119-1022b58d43c7 None admin] RBAC: Authorizing `identity:get_project(project_id=c0336e9e466e4bd4a28254482978f7b2)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:51.318294 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ceba4-d295-4f53-9119-1022b58d43c7 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:51.319999 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 219/443] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:51 2022] GET /identity/v3/projects/c0336e9e466e4bd4a28254482978f7b2 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:51.321232 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d06ab-7b10-4c00-be2b-5ab6ecc667cf None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:51.321523 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d06ab-7b10-4c00-be2b-5ab6ecc667cf None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:51.321800 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d06ab-7b10-4c00-be2b-5ab6ecc667cf None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:51.327244 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 220/444] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:51.328030 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d06ab-7b10-4c00-be2b-5ab6ecc667cf None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:51.328335 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba9407-6f4a-45c4-b4f4-1e797bebb116 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:51.362178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fcec0f-965c-4622-8fa2-3db82ad8ed63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:51.363070 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcec0f-965c-4622-8fa2-3db82ad8ed63 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:51.363360 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcec0f-965c-4622-8fa2-3db82ad8ed63 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:51.363641 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcec0f-965c-4622-8fa2-3db82ad8ed63 None admin] PATH_INFO: `/v3/groups/54b2eb315544408990a4259d8a76c132` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:51.364821 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fcec0f-965c-4622-8fa2-3db82ad8ed63 None admin] RBAC: Authorizing `identity:get_group(group_id=54b2eb315544408990a4259d8a76c132)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:51.366285 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fcec0f-965c-4622-8fa2-3db82ad8ed63 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:51.368099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 225/445] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jun 23 09:03:51 2022] GET /identity/v3/groups/54b2eb315544408990a4259d8a76c132 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:51.369071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb9a4e-ce1d-43d0-bb6e-b2a11843f37e None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:51.369360 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb9a4e-ce1d-43d0-bb6e-b2a11843f37e None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:51.369637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb9a4e-ce1d-43d0-bb6e-b2a11843f37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:51.386087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-11cb9a4e-ce1d-43d0-bb6e-b2a11843f37e None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:51.387365 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-11cb9a4e-ce1d-43d0-bb6e-b2a11843f37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:51.432822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d06ab-7b10-4c00-be2b-5ab6ecc667cf None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:51.458327 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:51.462894 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 226/446] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:51.463538 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cb9a4e-ce1d-43d0-bb6e-b2a11843f37e None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:51.474252 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa278292-e408-45c8-b9c6-ee73c698e92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:51.477938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa278292-e408-45c8-b9c6-ee73c698e92b None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:51.477938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa278292-e408-45c8-b9c6-ee73c698e92b None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:51.477938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa278292-e408-45c8-b9c6-ee73c698e92b None admin] PATH_INFO: `/v3/groups/54b2eb315544408990a4259d8a76c132` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:51.477938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa278292-e408-45c8-b9c6-ee73c698e92b None admin] RBAC: Authorizing `identity:get_group(group_id=54b2eb315544408990a4259d8a76c132)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:51.478324 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa278292-e408-45c8-b9c6-ee73c698e92b None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:51.480034 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 221/447] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jun 23 09:03:51 2022] GET /identity/v3/groups/54b2eb315544408990a4259d8a76c132 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:51.485578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa278292-e408-45c8-b9c6-ee73c698e92b None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:51.547148 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:51.561248 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cb9a4e-ce1d-43d0-bb6e-b2a11843f37e None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:51.592904 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa278292-e408-45c8-b9c6-ee73c698e92b None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:51.594822 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44892cd-3f66-44ec-8e80-5fc704cd00af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:51.595506 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44892cd-3f66-44ec-8e80-5fc704cd00af None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:51.595793 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44892cd-3f66-44ec-8e80-5fc704cd00af None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:51.596071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44892cd-3f66-44ec-8e80-5fc704cd00af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:51.596939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44892cd-3f66-44ec-8e80-5fc704cd00af None admin] RBAC: Adding query filter params (group.id=54b2eb315544408990a4259d8a76c132, role.id=53a3d65343394e7caafcc33f12cf138b, scope.project.id=c0336e9e466e4bd4a28254482978f7b2) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:51.597253 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44892cd-3f66-44ec-8e80-5fc704cd00af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:51.598689 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44892cd-3f66-44ec-8e80-5fc704cd00af None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:51.604686 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 227/448] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jun 23 09:03:51 2022] GET /identity/v3/role_assignments?group.id=54b2eb315544408990a4259d8a76c132&scope.project.id=c0336e9e466e4bd4a28254482978f7b2&role.id=53a3d65343394e7caafcc33f12cf138b => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:51.605451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44892cd-3f66-44ec-8e80-5fc704cd00af None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:51.629375 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37b093-d66e-43b3-a1bf-f52bee831a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:51.630059 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37b093-d66e-43b3-a1bf-f52bee831a83 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:51.630346 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37b093-d66e-43b3-a1bf-f52bee831a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:51.630952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37b093-d66e-43b3-a1bf-f52bee831a83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:51.631518 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37b093-d66e-43b3-a1bf-f52bee831a83 None admin] RBAC: Adding query filter params (group.id=54b2eb315544408990a4259d8a76c132, role.id=478b327fd22845cc98a2fafe821b1e85, scope.project.id=c0336e9e466e4bd4a28254482978f7b2) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:51.631929 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37b093-d66e-43b3-a1bf-f52bee831a83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:51.633346 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37b093-d66e-43b3-a1bf-f52bee831a83 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:51.650987 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 222/449] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jun 23 09:03:51 2022] GET /identity/v3/role_assignments?group.id=54b2eb315544408990a4259d8a76c132&scope.project.id=c0336e9e466e4bd4a28254482978f7b2&role.id=478b327fd22845cc98a2fafe821b1e85 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:51.698379 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:51.751559 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44892cd-3f66-44ec-8e80-5fc704cd00af None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:51.792249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55150756-8a9a-4c8c-89cb-d30667046602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:51.793066 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55150756-8a9a-4c8c-89cb-d30667046602 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:51.793916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55150756-8a9a-4c8c-89cb-d30667046602 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:51.794200 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55150756-8a9a-4c8c-89cb-d30667046602 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:51.806701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55150756-8a9a-4c8c-89cb-d30667046602 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:51.815368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-55150756-8a9a-4c8c-89cb-d30667046602 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:03:51.815368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:51.815368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:51.815368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:51.815368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:51.815368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:51.815368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:51.815368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:51.815368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:51.815368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:51.815368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:51.815368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:51.815368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:51.815368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:51.815368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:51.815368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:51.816317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:51.816317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:51.816317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:51.816317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:51.816317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:51.816317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:51.816317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:51.816317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:51.816317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:51.816317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:51.816317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:51.816317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:51.816317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:51.816317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:51.816317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:51.816317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:51.817178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:51.817178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:51.817178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:51.817178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:51.817178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:51.817178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:51.817178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:51.817178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:51.817178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:51.817178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:51.817178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:03:51.817178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:51.817178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 228/450] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:03:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:51.822468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37b093-d66e-43b3-a1bf-f52bee831a83 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:51.932170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37b093-d66e-43b3-a1bf-f52bee831a83 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:51.956722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:51.967547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bee977-810b-4111-9a5f-77f3dcdc29db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:51.970075 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bee977-810b-4111-9a5f-77f3dcdc29db None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:51.970075 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bee977-810b-4111-9a5f-77f3dcdc29db None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:51.970075 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bee977-810b-4111-9a5f-77f3dcdc29db None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:51.970075 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bee977-810b-4111-9a5f-77f3dcdc29db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:51.970075 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bee977-810b-4111-9a5f-77f3dcdc29db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:51.971245 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bee977-810b-4111-9a5f-77f3dcdc29db None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:51.979796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 223/451] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:03:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:51.986254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55150756-8a9a-4c8c-89cb-d30667046602 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:52.093877 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55150756-8a9a-4c8c-89cb-d30667046602 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:52.124537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:52.135056 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce4371-3e91-409d-97e2-0d4cf590e0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:52.135770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce4371-3e91-409d-97e2-0d4cf590e0af None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:52.136058 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce4371-3e91-409d-97e2-0d4cf590e0af None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:52.136388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce4371-3e91-409d-97e2-0d4cf590e0af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:52.203170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce4371-3e91-409d-97e2-0d4cf590e0af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:52.204631 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce4371-3e91-409d-97e2-0d4cf590e0af None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:52.300275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 229/452] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:03:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 09:03:52.305949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bee977-810b-4111-9a5f-77f3dcdc29db None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:52.402167 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bee977-810b-4111-9a5f-77f3dcdc29db None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:52.434486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759cc9b8-0508-43e6-8212-e6bd6fec2dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:52.435377 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759cc9b8-0508-43e6-8212-e6bd6fec2dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:52.435658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759cc9b8-0508-43e6-8212-e6bd6fec2dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:52.435932 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759cc9b8-0508-43e6-8212-e6bd6fec2dd8 None admin] PATH_INFO: `/v3/projects/e5fa9b7cbdd44d0c8b5ffa0c15d81646` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:52.437103 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759cc9b8-0508-43e6-8212-e6bd6fec2dd8 None admin] RBAC: Authorizing `identity:get_project(project_id=e5fa9b7cbdd44d0c8b5ffa0c15d81646)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:52.438533 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759cc9b8-0508-43e6-8212-e6bd6fec2dd8 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:52.440237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 224/453] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:03:52 2022] GET /identity/v3/projects/e5fa9b7cbdd44d0c8b5ffa0c15d81646 => generated 344 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:52.445708 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce4371-3e91-409d-97e2-0d4cf590e0af None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:52.470115 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:52.542958 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce4371-3e91-409d-97e2-0d4cf590e0af None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:52.575901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d42e8-87e6-44e3-b88e-d3f473613bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:52.576748 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d42e8-87e6-44e3-b88e-d3f473613bef None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:52.577035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d42e8-87e6-44e3-b88e-d3f473613bef None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:52.577312 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d42e8-87e6-44e3-b88e-d3f473613bef None admin] PATH_INFO: `/v3/groups/cc75182f2829402a8a6cb50cef5da0d6` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:52.578477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d42e8-87e6-44e3-b88e-d3f473613bef None admin] RBAC: Authorizing `identity:get_group(group_id=cc75182f2829402a8a6cb50cef5da0d6)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:52.579945 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d42e8-87e6-44e3-b88e-d3f473613bef None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:52.581557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 230/454] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jun 23 09:03:52 2022] GET /identity/v3/groups/cc75182f2829402a8a6cb50cef5da0d6 => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:52.587671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759cc9b8-0508-43e6-8212-e6bd6fec2dd8 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:52.641803 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:52.685076 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759cc9b8-0508-43e6-8212-e6bd6fec2dd8 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:52.717831 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edb0bb-efdb-4d4a-8e17-441f72e5494a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:52.718482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edb0bb-efdb-4d4a-8e17-441f72e5494a None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:52.718771 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edb0bb-efdb-4d4a-8e17-441f72e5494a None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:52.719079 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edb0bb-efdb-4d4a-8e17-441f72e5494a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:52.719914 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7edb0bb-efdb-4d4a-8e17-441f72e5494a None admin] RBAC: Adding query filter params (group.id=cc75182f2829402a8a6cb50cef5da0d6, role.id=9cd080b50c6f4ee8a54bf9ef1655bd83, scope.project.id=e5fa9b7cbdd44d0c8b5ffa0c15d81646) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:52.720229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7edb0bb-efdb-4d4a-8e17-441f72e5494a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:52.721662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7edb0bb-efdb-4d4a-8e17-441f72e5494a None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:52.727335 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 225/455] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jun 23 09:03:52 2022] GET /identity/v3/role_assignments?group.id=cc75182f2829402a8a6cb50cef5da0d6&scope.project.id=e5fa9b7cbdd44d0c8b5ffa0c15d81646&role.id=9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:55.299363 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f505b8-2e5b-41ec-85ea-1de80d580c4d None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:55.299363 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f505b8-2e5b-41ec-85ea-1de80d580c4d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:55.299363 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f505b8-2e5b-41ec-85ea-1de80d580c4d None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:55.300250 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 231/456] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:55.305738 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec74508-e9da-4080-b783-c640559f4696 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:55.306555 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec74508-e9da-4080-b783-c640559f4696 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:55.307093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec74508-e9da-4080-b783-c640559f4696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:55.331912 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-eec74508-e9da-4080-b783-c640559f4696 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:55.333727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-eec74508-e9da-4080-b783-c640559f4696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:55.424575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 226/457] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:55.443421 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9cb04-de08-4187-9928-f4935ff7b942 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:55.443713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9cb04-de08-4187-9928-f4935ff7b942 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:55.443994 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9cb04-de08-4187-9928-f4935ff7b942 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:55.445361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 232/458] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:55.450471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8addbf93-3450-4a20-8047-61d8006efe8c None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:55.450767 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8addbf93-3450-4a20-8047-61d8006efe8c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:55.451073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8addbf93-3450-4a20-8047-61d8006efe8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:55.470123 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9cb04-de08-4187-9928-f4935ff7b942 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:55.472701 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-8addbf93-3450-4a20-8047-61d8006efe8c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:55.473927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-8addbf93-3450-4a20-8047-61d8006efe8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:55.541867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:55.566211 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 227/459] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:55.567937 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d1b9c-f723-4397-b956-d9a75f506c57 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:55.568442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d1b9c-f723-4397-b956-d9a75f506c57 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:55.568946 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d1b9c-f723-4397-b956-d9a75f506c57 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:55.570449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 228/460] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:55.576876 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301abd6-6b4d-421e-89ce-3d65fc306fcd None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:55.577166 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301abd6-6b4d-421e-89ce-3d65fc306fcd None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:55.577448 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301abd6-6b4d-421e-89ce-3d65fc306fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:55.595957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-7301abd6-6b4d-421e-89ce-3d65fc306fcd None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:55.597204 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-7301abd6-6b4d-421e-89ce-3d65fc306fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:55.599773 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:55.613651 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9cb04-de08-4187-9928-f4935ff7b942 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:55.651576 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a5385-a149-4a37-80cb-8991ebc66f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:55.652791 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a5385-a149-4a37-80cb-8991ebc66f7a None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:55.653307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a5385-a149-4a37-80cb-8991ebc66f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:55.653788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a5385-a149-4a37-80cb-8991ebc66f7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:55.664402 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a5385-a149-4a37-80cb-8991ebc66f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:55.666164 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a5385-a149-4a37-80cb-8991ebc66f7a None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:55.679117 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 229/461] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:03:55.679992 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301abd6-6b4d-421e-89ce-3d65fc306fcd None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:55.686553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-cb9a5385-a149-4a37-80cb-8991ebc66f7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:03:55.686553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:55.686553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:55.686553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:55.686553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:55.686553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:55.686553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:55.686553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:55.686553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:55.686553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:55.686553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:55.686553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:55.686553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:03:55.686553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:03:55.686553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:03:55.687542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:03:55.687542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:55.687542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:55.687542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:55.687542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:55.687542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:55.687542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:55.687542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:55.687542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:55.687542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:55.687542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:55.687542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:55.687542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:55.687542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:55.687542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:55.687542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:55.687542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:55.688453 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:03:55.688453 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:03:55.688453 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:03:55.688453 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:55.690688 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 233/462] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:03:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:03:55.700381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a5385-a149-4a37-80cb-8991ebc66f7a None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:55.812523 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301abd6-6b4d-421e-89ce-3d65fc306fcd None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:55.819781 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a5385-a149-4a37-80cb-8991ebc66f7a None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:55.823090 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:55.849187 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fe2cf-a6eb-4586-a155-4e688eef39ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:55.850395 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fe2cf-a6eb-4586-a155-4e688eef39ca None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:55.850893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fe2cf-a6eb-4586-a155-4e688eef39ca None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:55.851425 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fe2cf-a6eb-4586-a155-4e688eef39ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:55.864956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765fe2cf-a6eb-4586-a155-4e688eef39ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:55.865835 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95819901-cc83-4768-96f5-32c9483a999c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:55.866604 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95819901-cc83-4768-96f5-32c9483a999c None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:55.867015 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95819901-cc83-4768-96f5-32c9483a999c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:55.867324 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765fe2cf-a6eb-4586-a155-4e688eef39ca None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:55.867440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95819901-cc83-4768-96f5-32c9483a999c None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:55.868269 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95819901-cc83-4768-96f5-32c9483a999c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:55.868673 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95819901-cc83-4768-96f5-32c9483a999c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:55.870771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95819901-cc83-4768-96f5-32c9483a999c None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:55.884477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 234/463] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:55.885399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95819901-cc83-4768-96f5-32c9483a999c None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:55.890562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-765fe2cf-a6eb-4586-a155-4e688eef39ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:03:55.890562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:55.890562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:55.890562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:55.890562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:55.890562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:55.890562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:55.890562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:55.890562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:55.890562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:55.890562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:55.890562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:55.890562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:03:55.890562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:03:55.890562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:03:55.891499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:03:55.891499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:55.891499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:55.891499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:55.891499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:55.891499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:55.891499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:55.891499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:55.891499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:55.891499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:55.891499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:55.891499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:55.891499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:55.891499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:55.891499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:55.891499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:55.891499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:55.892432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:03:55.892432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:03:55.892432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:03:55.892432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:55.894263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 230/464] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:03:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:55.895271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:55.895385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fe2cf-a6eb-4586-a155-4e688eef39ca None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:56.005674 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fe2cf-a6eb-4586-a155-4e688eef39ca None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:56.014851 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95819901-cc83-4768-96f5-32c9483a999c None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:56.040878 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005cff9-a49f-4577-9f09-719154e1b9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:56.041538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005cff9-a49f-4577-9f09-719154e1b9ef None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:56.041993 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005cff9-a49f-4577-9f09-719154e1b9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:56.042274 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005cff9-a49f-4577-9f09-719154e1b9ef None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:56.043045 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005cff9-a49f-4577-9f09-719154e1b9ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:56.044781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005cff9-a49f-4577-9f09-719154e1b9ef None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:56.053517 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc74d3d-3d23-4a88-8116-6dded532e87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:56.054872 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc74d3d-3d23-4a88-8116-6dded532e87e None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:56.055314 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc74d3d-3d23-4a88-8116-6dded532e87e None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:56.055607 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc74d3d-3d23-4a88-8116-6dded532e87e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:56.066631 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc74d3d-3d23-4a88-8116-6dded532e87e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:56.068224 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc74d3d-3d23-4a88-8116-6dded532e87e None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:56.085867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-3bc74d3d-3d23-4a88-8116-6dded532e87e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:03:56.085867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:56.085867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:56.085867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:56.085867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:56.085867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:56.085867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:56.085867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:56.085867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:56.085867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:56.085867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:56.085867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:56.085867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:03:56.085867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:03:56.085867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:03:56.086996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:03:56.086996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:56.086996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:56.086996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:56.086996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:56.086996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:56.086996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:56.086996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:56.086996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:56.086996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:56.086996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:56.086996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:56.086996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:56.086996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:56.086996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:56.086996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:56.086996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:56.087958 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:03:56.087958 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:03:56.087958 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:03:56.087958 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:56.087958 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 235/465] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:03:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:56.088259 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc74d3d-3d23-4a88-8116-6dded532e87e None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:56.138899 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 231/466] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jun 23 09:03:55 2022] POST /identity/v3/users => generated 246 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:03:56.144605 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45530fa5-9c4e-4dbc-94d7-ca472204574d None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:56.144911 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45530fa5-9c4e-4dbc-94d7-ca472204574d None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:56.145181 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45530fa5-9c4e-4dbc-94d7-ca472204574d None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:56.146835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 232/467] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:56.152845 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45530fa5-9c4e-4dbc-94d7-ca472204574d None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:56.159683 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:56.178866 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:56.217622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc74d3d-3d23-4a88-8116-6dded532e87e None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:56.303091 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c01aa-b1c9-45ec-8dbe-387812df1e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:56.303091 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c01aa-b1c9-45ec-8dbe-387812df1e4a None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:56.303091 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c01aa-b1c9-45ec-8dbe-387812df1e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:56.303091 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c01aa-b1c9-45ec-8dbe-387812df1e4a None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:56.303926 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c01aa-b1c9-45ec-8dbe-387812df1e4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:56.303926 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c01aa-b1c9-45ec-8dbe-387812df1e4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:56.303926 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c01aa-b1c9-45ec-8dbe-387812df1e4a None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:56.303926 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 236/468] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:56.303926 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f9a87-688d-4f51-9626-b4bf417c8ef9 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:56.303926 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f9a87-688d-4f51-9626-b4bf417c8ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:56.304304 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f9a87-688d-4f51-9626-b4bf417c8ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:56.304375 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45530fa5-9c4e-4dbc-94d7-ca472204574d None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:56.304471 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-c70f9a87-688d-4f51-9626-b4bf417c8ef9 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:56.304471 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-c70f9a87-688d-4f51-9626-b4bf417c8ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:56.344300 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9df31-01f8-4cc6-8c13-0d270464ea46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:56.345917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9df31-01f8-4cc6-8c13-0d270464ea46 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:56.346357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9df31-01f8-4cc6-8c13-0d270464ea46 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:56.346788 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9df31-01f8-4cc6-8c13-0d270464ea46 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:56.350081 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9df31-01f8-4cc6-8c13-0d270464ea46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:56.350569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9df31-01f8-4cc6-8c13-0d270464ea46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:56.352163 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9df31-01f8-4cc6-8c13-0d270464ea46 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:56.374007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 233/469] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:56.375109 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9df31-01f8-4cc6-8c13-0d270464ea46 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:56.399130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 237/470] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:56.399956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f9a87-688d-4f51-9626-b4bf417c8ef9 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:56.508231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:56.511271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:56.533824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9df31-01f8-4cc6-8c13-0d270464ea46 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:56.563692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f9a87-688d-4f51-9626-b4bf417c8ef9 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:56.571266 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d391d5-0755-4dfe-8e54-71576a194709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:56.571959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d391d5-0755-4dfe-8e54-71576a194709 None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:56.572246 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d391d5-0755-4dfe-8e54-71576a194709 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:56.572528 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d391d5-0755-4dfe-8e54-71576a194709 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:56.573252 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d391d5-0755-4dfe-8e54-71576a194709 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:56.575530 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d391d5-0755-4dfe-8e54-71576a194709 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:56.600982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6fb18-ef0b-400d-888b-d2f316e82969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:56.601661 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6fb18-ef0b-400d-888b-d2f316e82969 None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:56.601948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6fb18-ef0b-400d-888b-d2f316e82969 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:56.602228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6fb18-ef0b-400d-888b-d2f316e82969 None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:56.602999 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6fb18-ef0b-400d-888b-d2f316e82969 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:56.604447 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6fb18-ef0b-400d-888b-d2f316e82969 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:56.653208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 234/471] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jun 23 09:03:56 2022] POST /identity/v3/users => generated 248 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:03:56.659290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d391d5-0755-4dfe-8e54-71576a194709 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:56.722228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 238/472] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jun 23 09:03:56 2022] POST /identity/v3/users => generated 248 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:03:56.736104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c806a-6eb7-49af-99a0-ab62cf25ee96 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:56.736499 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c806a-6eb7-49af-99a0-ab62cf25ee96 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:56.736715 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c806a-6eb7-49af-99a0-ab62cf25ee96 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:56.738463 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 239/473] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:56.752784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29086be-764a-4f39-b8bd-e59c901e98d7 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:56.753150 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29086be-764a-4f39-b8bd-e59c901e98d7 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:56.753364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29086be-764a-4f39-b8bd-e59c901e98d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:56.781117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-b29086be-764a-4f39-b8bd-e59c901e98d7 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:56.782787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b29086be-764a-4f39-b8bd-e59c901e98d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:56.790670 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d391d5-0755-4dfe-8e54-71576a194709 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:56.824625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:56.831326 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee9c02-c6af-4949-bc2a-9a7e03bb50d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:56.832578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee9c02-c6af-4949-bc2a-9a7e03bb50d8 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:56.833370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee9c02-c6af-4949-bc2a-9a7e03bb50d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:56.833929 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee9c02-c6af-4949-bc2a-9a7e03bb50d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:56.847804 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee9c02-c6af-4949-bc2a-9a7e03bb50d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:56.849690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee9c02-c6af-4949-bc2a-9a7e03bb50d8 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:56.871006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-bcee9c02-c6af-4949-bc2a-9a7e03bb50d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:03:56.871006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:56.871006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:56.871006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:56.871006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:56.871006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:56.871006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:56.871006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:56.871006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:56.871006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:56.871006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:56.871006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:56.871006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:03:56.871006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:03:56.871006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:03:56.872504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:03:56.872504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:56.872504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:56.872504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:56.872504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:56.872504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:56.872504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:56.872504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:56.872504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:56.872504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:56.872504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:56.872504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:56.872504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:56.872504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:56.872504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:56.872504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:56.872504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:56.873453 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:03:56.873453 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:03:56.873453 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:03:56.873453 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:56.875851 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 235/474] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:03:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:56.890018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee9c02-c6af-4949-bc2a-9a7e03bb50d8 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:56.901333 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 240/475] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:56.934625 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29086be-764a-4f39-b8bd-e59c901e98d7 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:56.972398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:57.025583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee9c02-c6af-4949-bc2a-9a7e03bb50d8 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:57.058596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29086be-764a-4f39-b8bd-e59c901e98d7 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:57.070295 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043c2be-9e1a-4cdc-968f-e6b1fc0fc3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:57.071132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043c2be-9e1a-4cdc-968f-e6b1fc0fc3fb None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:57.071406 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043c2be-9e1a-4cdc-968f-e6b1fc0fc3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:57.071679 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043c2be-9e1a-4cdc-968f-e6b1fc0fc3fb None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:57.072454 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043c2be-9e1a-4cdc-968f-e6b1fc0fc3fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:57.072799 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043c2be-9e1a-4cdc-968f-e6b1fc0fc3fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:57.074185 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043c2be-9e1a-4cdc-968f-e6b1fc0fc3fb None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:57.089654 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 236/476] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:03:57.095429 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043c2be-9e1a-4cdc-968f-e6b1fc0fc3fb None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:57.110063 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3eca71-9f45-42ce-9a7b-30541ce148f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:57.111154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3eca71-9f45-42ce-9a7b-30541ce148f9 None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:57.111490 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3eca71-9f45-42ce-9a7b-30541ce148f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:57.111767 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3eca71-9f45-42ce-9a7b-30541ce148f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:57.112534 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3eca71-9f45-42ce-9a7b-30541ce148f9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:57.113925 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3eca71-9f45-42ce-9a7b-30541ce148f9 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:57.149607 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:57.163601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.sql.core [None req-fa3eca71-9f45-42ce-9a7b-30541ce148f9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 09:03:57.163601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 09:03:57.163601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [parameters: {'id': 'd2b56322faff4e75b27114ceede0f114', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 09:03:57.163601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80655) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 09:03:57.204072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-fa3eca71-9f45-42ce-9a7b-30541ce148f9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 09:03:57.204072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:57.204072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 09:03:57.204072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 09:03:57.204072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 09:03:57.204072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 09:03:57.204072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 09:03:57.204072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 09:03:57.204072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 09:03:57.204072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application conn.query(q) Jun 23 09:03:57.204072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 09:03:57.204072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 09:03:57.204072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 09:03:57.204072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application result.read() Jun 23 09:03:57.204072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 09:03:57.205057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 09:03:57.205057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 09:03:57.205057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 09:03:57.205057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 09:03:57.205057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 09:03:57.205057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 09:03:57.205057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 09:03:57.205057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 09:03:57.205057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:57.205057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 09:03:57.205057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:57.205057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:57.205057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 09:03:57.205057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 09:03:57.205057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 23 09:03:57.205057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return ref.to_dict() Jun 23 09:03:57.205057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 09:03:57.205057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application next(self.gen) Jun 23 09:03:57.206021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 09:03:57.206021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application yield resource Jun 23 09:03:57.206021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 09:03:57.206021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application next(self.gen) Jun 23 09:03:57.206021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 09:03:57.206021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 09:03:57.206021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 09:03:57.206021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 09:03:57.206021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 09:03:57.206021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise self.value Jun 23 09:03:57.206021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 09:03:57.206021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 09:03:57.206021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 09:03:57.206021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application session.commit() Jun 23 09:03:57.206021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 09:03:57.206021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 09:03:57.206021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 09:03:57.206986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 09:03:57.206986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 09:03:57.206986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self.session.flush() Jun 23 09:03:57.206986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 09:03:57.206986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 09:03:57.206986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 09:03:57.206986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 09:03:57.206986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 09:03:57.206986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application compat.raise_( Jun 23 09:03:57.206986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 09:03:57.206986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception Jun 23 09:03:57.206986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 09:03:57.206986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 09:03:57.206986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 09:03:57.206986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 09:03:57.206986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 09:03:57.206986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 09:03:57.207962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 09:03:57.207962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 09:03:57.207962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 09:03:57.207962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 09:03:57.207962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 09:03:57.207962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 09:03:57.207962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 09:03:57.207962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 09:03:57.207962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 09:03:57.207962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 09:03:57.207962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 09:03:57.207962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 09:03:57.207962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 09:03:57.207962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 09:03:57.207962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 09:03:57.207962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception Jun 23 09:03:57.209109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 09:03:57.209109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 09:03:57.209109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 09:03:57.209109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 09:03:57.209109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 09:03:57.209109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 09:03:57.209109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 09:03:57.209109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application conn.query(q) Jun 23 09:03:57.209109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 09:03:57.209109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 09:03:57.209109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 09:03:57.209109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application result.read() Jun 23 09:03:57.209109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 09:03:57.209109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 09:03:57.209109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 09:03:57.209109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 09:03:57.209932 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 09:03:57.209932 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 09:03:57.209932 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 09:03:57.209932 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 09:03:57.209932 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 09:03:57.209932 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 09:03:57.209932 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [parameters: {'id': 'd2b56322faff4e75b27114ceede0f114', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 09:03:57.209932 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 09:03:57.209932 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:57.209932 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:03:57.209932 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:57.209932 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:57.209932 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:57.209932 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:57.209932 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:57.209932 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:57.212951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:57.212951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:57.212951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:57.212951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:57.212951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:57.212951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:57.212951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 23 09:03:57.212951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 23 09:03:57.212951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:57.212951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:57.212951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 23 09:03:57.212951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 23 09:03:57.212951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 09:03:57.212951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 09:03:57.212951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 09:03:57.212951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:57.215043 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 241/477] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Jun 23 09:03:56 2022] POST /identity/v3/roles => generated 144 bytes in 272 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:03:57.215700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043c2be-9e1a-4cdc-968f-e6b1fc0fc3fb None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:57.219025 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3eca71-9f45-42ce-9a7b-30541ce148f9 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:57.249446 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851e132-25d5-4145-9432-79f76f2dbe54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:57.250281 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851e132-25d5-4145-9432-79f76f2dbe54 None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:57.250574 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851e132-25d5-4145-9432-79f76f2dbe54 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:57.251003 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851e132-25d5-4145-9432-79f76f2dbe54 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:57.251735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851e132-25d5-4145-9432-79f76f2dbe54 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:57.253541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851e132-25d5-4145-9432-79f76f2dbe54 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:57.354602 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:57.358045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3eca71-9f45-42ce-9a7b-30541ce148f9 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:57.367411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 237/478] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jun 23 09:03:57 2022] POST /identity/v3/users => generated 249 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:03:57.396589 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91c2f5-a772-4dec-a84f-ced25fa6f834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:57.397675 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91c2f5-a772-4dec-a84f-ced25fa6f834 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:57.398126 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91c2f5-a772-4dec-a84f-ced25fa6f834 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:57.398592 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91c2f5-a772-4dec-a84f-ced25fa6f834 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:57.411145 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab91c2f5-a772-4dec-a84f-ced25fa6f834 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:57.419904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-ab91c2f5-a772-4dec-a84f-ced25fa6f834 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:57.419904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:57.419904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:57.419904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:57.419904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:57.419904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:57.419904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:57.419904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:57.419904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:57.419904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:57.419904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:57.419904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:57.419904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:57.419904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:57.419904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:57.419904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:57.421104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:57.421104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:57.421104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:57.421104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:57.421104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:57.421104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:57.421104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:57.421104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:57.421104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:57.421104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:57.421104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:57.421104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:57.421104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:57.421104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:57.421104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:57.421104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:57.422138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:57.422138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:57.422138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:57.422138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:57.422138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:57.422138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:57.422138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:57.422138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:57.422138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:57.422138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:57.422138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 09:03:57.422138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:57.424640 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 242/479] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:03:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:57.430381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851e132-25d5-4145-9432-79f76f2dbe54 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:57.540866 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:57.544149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851e132-25d5-4145-9432-79f76f2dbe54 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:57.581677 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8408ceb8-2ee0-4162-9e79-dc409a069d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:57.582368 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408ceb8-2ee0-4162-9e79-dc409a069d2f None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:57.582655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408ceb8-2ee0-4162-9e79-dc409a069d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:57.582970 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408ceb8-2ee0-4162-9e79-dc409a069d2f None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:57.584055 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8408ceb8-2ee0-4162-9e79-dc409a069d2f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:57.584365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8408ceb8-2ee0-4162-9e79-dc409a069d2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:57.585774 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8408ceb8-2ee0-4162-9e79-dc409a069d2f None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:57.595719 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 238/480] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:03:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:58.590826 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dadf45-5a78-45d6-8b51-8c37ec84e21e None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:58.591408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dadf45-5a78-45d6-8b51-8c37ec84e21e None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:58.591408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dadf45-5a78-45d6-8b51-8c37ec84e21e None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:58.593031 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 243/481] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:58.603733 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b14f4-b73e-45dd-b781-94ce516b4710 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:58.604146 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b14f4-b73e-45dd-b781-94ce516b4710 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:58.604329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b14f4-b73e-45dd-b781-94ce516b4710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:58.621129 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-ec3b14f4-b73e-45dd-b781-94ce516b4710 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:58.622370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ec3b14f4-b73e-45dd-b781-94ce516b4710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:58.713175 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 239/482] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:58.752390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dadf45-5a78-45d6-8b51-8c37ec84e21e None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:58.883749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dadf45-5a78-45d6-8b51-8c37ec84e21e None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:58.929945 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf32150-3fb3-45fe-8f9e-9e15539367e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:58.930752 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf32150-3fb3-45fe-8f9e-9e15539367e0 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:58.931064 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf32150-3fb3-45fe-8f9e-9e15539367e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:58.931340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf32150-3fb3-45fe-8f9e-9e15539367e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:58.948144 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf32150-3fb3-45fe-8f9e-9e15539367e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:58.956212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-dcf32150-3fb3-45fe-8f9e-9e15539367e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:03:58.956212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:58.956212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:58.956212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:58.956212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:58.956212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:58.956212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:58.956212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:58.956212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:58.956212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:58.956212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:58.956212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:58.956212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:58.956212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:58.956212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:58.956212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:58.957243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:58.957243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:58.957243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:58.957243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:58.957243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:58.957243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:58.957243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:58.957243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:58.957243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:58.957243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:58.957243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:58.957243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:58.957243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:58.957243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:58.957243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:58.957243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:58.958801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:58.958801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:58.958801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:58.958801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:58.958801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:58.958801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:58.958801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:58.958801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:58.958801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:58.958801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:58.958801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:03:58.958801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:58.958801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 244/483] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:03:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:58.964884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b14f4-b73e-45dd-b781-94ce516b4710 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:59.075427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:59.088095 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b14f4-b73e-45dd-b781-94ce516b4710 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:59.132563 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e379ac-c733-4eea-ac5d-cfb07524facd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:59.133255 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e379ac-c733-4eea-ac5d-cfb07524facd None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:59.133544 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e379ac-c733-4eea-ac5d-cfb07524facd None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:59.133875 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e379ac-c733-4eea-ac5d-cfb07524facd None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:59.134582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e379ac-c733-4eea-ac5d-cfb07524facd None admin] RBAC: Adding query filter params (name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:59.134891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e379ac-c733-4eea-ac5d-cfb07524facd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:59.136310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e379ac-c733-4eea-ac5d-cfb07524facd None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:59.145325 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 240/484] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jun 23 09:03:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:59.155882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf32150-3fb3-45fe-8f9e-9e15539367e0 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:59.186006 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:59.289183 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf32150-3fb3-45fe-8f9e-9e15539367e0 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:59.337990 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c633f0-a4b5-42eb-a0f3-96d29e2aa5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:59.338790 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c633f0-a4b5-42eb-a0f3-96d29e2aa5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:59.339111 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c633f0-a4b5-42eb-a0f3-96d29e2aa5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:59.339387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c633f0-a4b5-42eb-a0f3-96d29e2aa5f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:59.350248 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c633f0-a4b5-42eb-a0f3-96d29e2aa5f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:59.351764 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c633f0-a4b5-42eb-a0f3-96d29e2aa5f2 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:59.380943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-f4c633f0-a4b5-42eb-a0f3-96d29e2aa5f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:03:59.380943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:59.380943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:59.380943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:59.380943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:59.380943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:59.380943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:59.380943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:59.380943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:59.380943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:59.380943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:59.380943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:59.380943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:03:59.380943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:03:59.380943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:03:59.381910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:03:59.381910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:59.381910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:59.381910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:59.381910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:59.381910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:59.381910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:59.381910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:59.381910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:59.381910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:59.381910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:59.381910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:59.381910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:59.381910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:59.381910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:59.381910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:59.381910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:59.383484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:03:59.383484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:03:59.383484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:03:59.383484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:59.383484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 245/485] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:03:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:59.388238 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e379ac-c733-4eea-ac5d-cfb07524facd None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:59.482407 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfd974-1e41-4a8d-b959-cc1f3fd48ebe None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:59.482772 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfd974-1e41-4a8d-b959-cc1f3fd48ebe None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:59.483023 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfd974-1e41-4a8d-b959-cc1f3fd48ebe None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:59.484650 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 246/486] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:59.489747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b7452-3341-4e28-abde-8798db34f943 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:59.489963 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b7452-3341-4e28-abde-8798db34f943 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:59.490248 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b7452-3341-4e28-abde-8798db34f943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:59.513128 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-342b7452-3341-4e28-abde-8798db34f943 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:59.516700 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-342b7452-3341-4e28-abde-8798db34f943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:59.577027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e379ac-c733-4eea-ac5d-cfb07524facd None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:59.595499 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:59.609610 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 247/487] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:59.633981 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b7452-3341-4e28-abde-8798db34f943 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:59.640121 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4ad42-683c-46b1-b809-709ce6ba39c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:59.642042 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4ad42-683c-46b1-b809-709ce6ba39c2 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:59.642042 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4ad42-683c-46b1-b809-709ce6ba39c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:59.642042 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4ad42-683c-46b1-b809-709ce6ba39c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:59.642042 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4ad42-683c-46b1-b809-709ce6ba39c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:03:59.644295 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4ad42-683c-46b1-b809-709ce6ba39c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:59.644295 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4ad42-683c-46b1-b809-709ce6ba39c2 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:59.661798 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 241/488] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:03:59.663297 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db123ce-4559-4d90-97f2-08323ae05041 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:59.663646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db123ce-4559-4d90-97f2-08323ae05041 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:59.663926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db123ce-4559-4d90-97f2-08323ae05041 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:59.665370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 242/489] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:03:59.667741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db123ce-4559-4d90-97f2-08323ae05041 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:59.687132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:59.752309 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b7452-3341-4e28-abde-8798db34f943 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:59.786713 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db123ce-4559-4d90-97f2-08323ae05041 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:03:59.788437 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429fefd-0230-48e9-8a2b-a831257a6a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:59.789243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429fefd-0230-48e9-8a2b-a831257a6a22 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:59.789568 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429fefd-0230-48e9-8a2b-a831257a6a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:59.789795 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429fefd-0230-48e9-8a2b-a831257a6a22 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:59.800967 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429fefd-0230-48e9-8a2b-a831257a6a22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:03:59.809465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-6429fefd-0230-48e9-8a2b-a831257a6a22 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:03:59.809465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:59.809465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:59.809465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:59.809465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:59.809465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:59.809465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:59.809465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:59.809465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:59.809465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:59.809465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:59.809465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:59.809465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:03:59.809465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:03:59.809465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:03:59.809465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:03:59.810414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:03:59.810414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:03:59.810414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:03:59.810414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:03:59.810414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:59.810414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:59.810414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:59.810414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:59.810414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:59.810414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:59.810414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:59.810414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:59.810414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:59.810414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:59.810414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:59.810414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:59.811329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:59.811329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:59.811329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:59.811329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:59.811329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:03:59.811329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:03:59.811329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:03:59.811329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:03:59.811329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:03:59.811329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:03:59.811329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:03:59.811329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:03:59.815171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 248/490] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:03:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:03:59.816488 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47280194-7e64-4328-b77e-9262ea36340f None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:59.816782 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47280194-7e64-4328-b77e-9262ea36340f None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:59.817066 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47280194-7e64-4328-b77e-9262ea36340f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:59.823306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3f5fe-62f6-47e7-b775-20b8b80edc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:03:59.824209 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3f5fe-62f6-47e7-b775-20b8b80edc7b None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:03:59.824567 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3f5fe-62f6-47e7-b775-20b8b80edc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:03:59.824850 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3f5fe-62f6-47e7-b775-20b8b80edc7b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:03:59.832460 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3f5fe-62f6-47e7-b775-20b8b80edc7b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:03:59.834171 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3f5fe-62f6-47e7-b775-20b8b80edc7b None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:03:59.835441 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-47280194-7e64-4328-b77e-9262ea36340f None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:03:59.836644 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-47280194-7e64-4328-b77e-9262ea36340f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:03:59.851229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-34e3f5fe-62f6-47e7-b775-20b8b80edc7b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 09:03:59.851229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:03:59.851229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:03:59.851229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:03:59.851229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:03:59.851229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:03:59.851229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:03:59.851229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:03:59.851229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:03:59.851229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:03:59.851229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:03:59.851229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:03:59.851229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:03:59.851229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:03:59.851229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:03:59.851229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:03:59.853536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:03:59.853536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:03:59.853536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:03:59.853536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:03:59.853536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:03:59.853536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:03:59.853536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:03:59.853536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:03:59.853536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:03:59.853536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:03:59.853536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:03:59.853536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:03:59.853536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:03:59.853536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:03:59.853536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:03:59.853536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:03:59.853536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:03:59.853536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:03:59.859575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:03:59.859575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:03:59.859575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:03:59.859575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:03:59.859575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:03:59.859575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:03:59.859575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:03:59.859575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:03:59.859575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 09:03:59.859575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:03:59.859575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 243/491] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jun 23 09:03:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:03:59.859575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3f5fe-62f6-47e7-b775-20b8b80edc7b None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:59.903329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:59.917420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 249/492] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:03:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:03:59.925040 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47280194-7e64-4328-b77e-9262ea36340f None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:03:59.937583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:03:59.964879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3f5fe-62f6-47e7-b775-20b8b80edc7b None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:00.004797 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1e582-0044-443b-8c70-2881cbff7573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:00.005711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1e582-0044-443b-8c70-2881cbff7573 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:00.006164 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1e582-0044-443b-8c70-2881cbff7573 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:00.006605 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1e582-0044-443b-8c70-2881cbff7573 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:00.007723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1e582-0044-443b-8c70-2881cbff7573 None admin] RBAC: Adding query filter params (name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:00.008273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1e582-0044-443b-8c70-2881cbff7573 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:00.009810 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1e582-0044-443b-8c70-2881cbff7573 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:00.019598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 244/493] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jun 23 09:03:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:00.020743 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1e582-0044-443b-8c70-2881cbff7573 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:00.028707 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47280194-7e64-4328-b77e-9262ea36340f None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:00.064392 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadeec7-ab03-4ce8-8d06-3dbe69020056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:00.065072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadeec7-ab03-4ce8-8d06-3dbe69020056 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:00.065356 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadeec7-ab03-4ce8-8d06-3dbe69020056 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:00.065703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadeec7-ab03-4ce8-8d06-3dbe69020056 None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:00.066545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aadeec7-ab03-4ce8-8d06-3dbe69020056 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:00.066856 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aadeec7-ab03-4ce8-8d06-3dbe69020056 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:00.073857 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aadeec7-ab03-4ce8-8d06-3dbe69020056 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:00.112433 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 250/494] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jun 23 09:03:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:00.112433 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadeec7-ab03-4ce8-8d06-3dbe69020056 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:00.135938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1e582-0044-443b-8c70-2881cbff7573 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:00.159861 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:00.182491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4006b70-6cd9-408f-94ad-6a8739a8326e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:00.183608 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4006b70-6cd9-408f-94ad-6a8739a8326e None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:00.183608 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4006b70-6cd9-408f-94ad-6a8739a8326e None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:00.184446 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4006b70-6cd9-408f-94ad-6a8739a8326e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:00.187294 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:00.196511 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4006b70-6cd9-408f-94ad-6a8739a8326e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:00.204426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-b4006b70-6cd9-408f-94ad-6a8739a8326e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:00.204426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:00.204426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:00.204426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:00.204426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:00.204426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:00.204426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:00.204426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:00.204426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:00.204426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:00.204426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:00.204426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:00.204426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:00.204426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:00.204426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:00.204426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:00.206823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:00.206823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:00.206823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:00.206823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:00.206823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:00.206823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:00.206823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:00.206823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:00.206823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:00.206823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:00.206823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:00.206823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:00.206823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:00.206823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:00.206823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:00.206823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:00.208088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:00.208088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:00.208088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:00.208088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:00.208088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:00.208088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:00.208088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:00.208088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:00.208088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:00.208088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:00.208088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:00.208088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:00.210693 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 245/495] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:04:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:00.211843 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f149f-d94c-43e3-b8bd-2e94bf556bc8 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:00.217118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f149f-d94c-43e3-b8bd-2e94bf556bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:00.217118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f149f-d94c-43e3-b8bd-2e94bf556bc8 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:00.217118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 246/496] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:00.217393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadeec7-ab03-4ce8-8d06-3dbe69020056 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:00.217473 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f149f-d94c-43e3-b8bd-2e94bf556bc8 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:00.250143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296dc5bd-f8f6-41f0-907a-d1e8f3bfe792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:00.251032 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296dc5bd-f8f6-41f0-907a-d1e8f3bfe792 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:00.251324 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296dc5bd-f8f6-41f0-907a-d1e8f3bfe792 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:00.251602 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296dc5bd-f8f6-41f0-907a-d1e8f3bfe792 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:00.262504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296dc5bd-f8f6-41f0-907a-d1e8f3bfe792 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:00.264234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296dc5bd-f8f6-41f0-907a-d1e8f3bfe792 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:00.281388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-296dc5bd-f8f6-41f0-907a-d1e8f3bfe792 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:00.281388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:00.281388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:00.281388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:00.281388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:00.281388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:00.281388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:00.281388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:00.281388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:00.281388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:00.281388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:00.281388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:00.281388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:00.281388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:00.281388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:00.282465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:00.282465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:00.282465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:00.282465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:00.282465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:00.282465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:00.282465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:00.282465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:00.282465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:00.282465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:00.282465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:00.282465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:00.282465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:00.282465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:00.282465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:00.282465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:00.282465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:00.283632 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:00.283632 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:00.283632 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:00.283632 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:00.283632 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 251/497] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:00.284334 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6a4f7-a7e3-44c5-92f5-987e097eeb25 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:00.284629 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6a4f7-a7e3-44c5-92f5-987e097eeb25 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:00.284907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6a4f7-a7e3-44c5-92f5-987e097eeb25 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:00.286172 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 252/498] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:00.287085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6a4f7-a7e3-44c5-92f5-987e097eeb25 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:00.322332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f149f-d94c-43e3-b8bd-2e94bf556bc8 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:00.356600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af74d9-0bc6-4d58-bc9b-5c369f381474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:00.357290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af74d9-0bc6-4d58-bc9b-5c369f381474 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:00.357584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af74d9-0bc6-4d58-bc9b-5c369f381474 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:00.357871 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af74d9-0bc6-4d58-bc9b-5c369f381474 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:00.391305 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:00.395864 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6a4f7-a7e3-44c5-92f5-987e097eeb25 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:00.417727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:00.424194 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87af74d9-0bc6-4d58-bc9b-5c369f381474 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:00.425684 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87af74d9-0bc6-4d58-bc9b-5c369f381474 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:00.428940 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a84ac-869c-4296-bd47-3d7cc9a1d8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:00.429608 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a84ac-869c-4296-bd47-3d7cc9a1d8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:00.429895 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a84ac-869c-4296-bd47-3d7cc9a1d8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:00.430173 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a84ac-869c-4296-bd47-3d7cc9a1d8a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:00.430939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a84ac-869c-4296-bd47-3d7cc9a1d8a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:00.431251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a84ac-869c-4296-bd47-3d7cc9a1d8a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:00.432616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a84ac-869c-4296-bd47-3d7cc9a1d8a5 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:00.441254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 253/499] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jun 23 09:04:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:00.442442 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966d5e4-1a66-49f6-81f5-8b240d4916be None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:00.442731 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966d5e4-1a66-49f6-81f5-8b240d4916be None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:00.443039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966d5e4-1a66-49f6-81f5-8b240d4916be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:00.459335 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-7966d5e4-1a66-49f6-81f5-8b240d4916be None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:00.460548 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7966d5e4-1a66-49f6-81f5-8b240d4916be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:00.528440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 254/500] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:04:00.529726 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966d5e4-1a66-49f6-81f5-8b240d4916be None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:00.529859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 247/501] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:04:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:04:00.534381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baef443-d5e0-4636-86b0-25b8fcac6b0a None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:00.534381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baef443-d5e0-4636-86b0-25b8fcac6b0a None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:00.534381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baef443-d5e0-4636-86b0-25b8fcac6b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:00.549203 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-6baef443-d5e0-4636-86b0-25b8fcac6b0a None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:00.550408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-6baef443-d5e0-4636-86b0-25b8fcac6b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:00.616847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 248/502] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:04:00.617687 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baef443-d5e0-4636-86b0-25b8fcac6b0a None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:00.628789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:00.638654 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966d5e4-1a66-49f6-81f5-8b240d4916be None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:00.657290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:00.672915 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2f310-dbcd-4a90-829d-84869c1d4c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:00.673529 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2f310-dbcd-4a90-829d-84869c1d4c1e None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:00.673897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2f310-dbcd-4a90-829d-84869c1d4c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:00.674179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2f310-dbcd-4a90-829d-84869c1d4c1e None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:00.674903 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2f310-dbcd-4a90-829d-84869c1d4c1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:00.675241 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2f310-dbcd-4a90-829d-84869c1d4c1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:00.676629 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2f310-dbcd-4a90-829d-84869c1d4c1e None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:00.688103 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 255/503] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:00.688883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2f310-dbcd-4a90-829d-84869c1d4c1e None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:00.731508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baef443-d5e0-4636-86b0-25b8fcac6b0a None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:00.764990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56242c1a-37f3-4242-841a-0945274d0349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:00.765785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56242c1a-37f3-4242-841a-0945274d0349 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:00.766069 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56242c1a-37f3-4242-841a-0945274d0349 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:00.766343 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56242c1a-37f3-4242-841a-0945274d0349 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:00.776162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56242c1a-37f3-4242-841a-0945274d0349 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:00.777726 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56242c1a-37f3-4242-841a-0945274d0349 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:00.785643 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2f310-dbcd-4a90-829d-84869c1d4c1e None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:00.795490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-56242c1a-37f3-4242-841a-0945274d0349 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:00.795490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:00.795490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:00.795490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:00.795490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:00.795490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:00.795490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:00.795490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:00.795490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:00.795490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:00.795490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:00.795490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:00.795490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:00.795490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:00.795490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:00.796408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:00.796408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:00.796408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:00.796408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:00.796408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:00.796408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:00.796408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:00.796408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:00.796408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:00.796408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:00.796408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:00.796408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:00.796408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:00.796408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:00.796408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:00.796408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:00.796408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:00.797380 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:00.797380 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:00.797380 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:00.797380 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:00.797380 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 249/504] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:00.797950 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56242c1a-37f3-4242-841a-0945274d0349 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:00.830822 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b02f47-5db2-44ff-8511-70546a8b15c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:00.831652 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b02f47-5db2-44ff-8511-70546a8b15c8 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:00.831939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b02f47-5db2-44ff-8511-70546a8b15c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:00.832216 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b02f47-5db2-44ff-8511-70546a8b15c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:00.843195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b02f47-5db2-44ff-8511-70546a8b15c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:00.844665 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b02f47-5db2-44ff-8511-70546a8b15c8 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:00.862272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-a3b02f47-5db2-44ff-8511-70546a8b15c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:00.862272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:00.862272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:00.862272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:00.862272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:00.862272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:00.862272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:00.862272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:00.862272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:00.862272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:00.862272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:00.862272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:00.862272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:00.862272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:00.862272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:00.863303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:00.863303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:00.863303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:00.863303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:00.863303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:00.863303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:00.863303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:00.863303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:00.863303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:00.863303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:00.863303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:00.863303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:00.863303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:00.863303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:00.863303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:00.863303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:00.863303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:00.864210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:00.864210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:00.864210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:00.864210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:00.864210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 256/505] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:00.864655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b02f47-5db2-44ff-8511-70546a8b15c8 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:00.866829 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:00.885752 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:00.906938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56242c1a-37f3-4242-841a-0945274d0349 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:00.940059 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec1f1f-3eb4-4ae4-aa02-0f088d7ef3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:00.940774 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec1f1f-3eb4-4ae4-aa02-0f088d7ef3da None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:00.941059 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec1f1f-3eb4-4ae4-aa02-0f088d7ef3da None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:00.941334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec1f1f-3eb4-4ae4-aa02-0f088d7ef3da None admin] PATH_INFO: `/v3/roles/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:00.952386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec1f1f-3eb4-4ae4-aa02-0f088d7ef3da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:00.960293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-baec1f1f-3eb4-4ae4-aa02-0f088d7ef3da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:00.960293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:00.960293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:00.960293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:00.960293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:00.960293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:00.960293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:00.960293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:00.960293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:00.960293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:00.960293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:00.960293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:00.960293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:00.960293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:00.960293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:00.960293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:00.961284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:00.961284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:00.961284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:00.961284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:00.961284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:00.961284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:00.961284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:00.961284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:00.961284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:00.961284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:00.961284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:00.961284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:00.961284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:00.961284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:00.961284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:00.961284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:00.962152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:00.962152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:00.962152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:00.962152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:00.962152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:00.962152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:00.962152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:00.962152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:00.962152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:00.962152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:00.962152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:00.962152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:00.962152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 250/506] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:04:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:00.962859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec1f1f-3eb4-4ae4-aa02-0f088d7ef3da None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:00.963275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b02f47-5db2-44ff-8511-70546a8b15c8 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:00.999983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d891c9-ca5e-4ab1-8ac4-a8f14e69e549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:01.000788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d891c9-ca5e-4ab1-8ac4-a8f14e69e549 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:01.001080 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d891c9-ca5e-4ab1-8ac4-a8f14e69e549 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:01.001363 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d891c9-ca5e-4ab1-8ac4-a8f14e69e549 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:01.012548 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d891c9-ca5e-4ab1-8ac4-a8f14e69e549 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:01.013997 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d891c9-ca5e-4ab1-8ac4-a8f14e69e549 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:01.030452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-66d891c9-ca5e-4ab1-8ac4-a8f14e69e549 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:01.030452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:01.030452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:01.030452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:01.030452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:01.030452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:01.030452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:01.030452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:01.030452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:01.030452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:01.030452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:01.030452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:01.030452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:01.030452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:01.030452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:01.031443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:01.031443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:01.031443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:01.031443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:01.031443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:01.031443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:01.031443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:01.031443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:01.031443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:01.031443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:01.031443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:01.031443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:01.031443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:01.031443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:01.031443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:01.031443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:01.031443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:01.032418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:01.032418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:01.032418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:01.032418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:01.032418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 257/507] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:01.032418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d891c9-ca5e-4ab1-8ac4-a8f14e69e549 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:01.060263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec1f1f-3eb4-4ae4-aa02-0f088d7ef3da None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:01.091084 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:01.094108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785f041-d74d-483c-9b36-2a634ebab021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:01.094950 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785f041-d74d-483c-9b36-2a634ebab021 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:01.095238 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785f041-d74d-483c-9b36-2a634ebab021 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:01.095517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785f041-d74d-483c-9b36-2a634ebab021 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:01.102884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785f041-d74d-483c-9b36-2a634ebab021 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:01.104407 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785f041-d74d-483c-9b36-2a634ebab021 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:01.106766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:01.119060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-f785f041-d74d-483c-9b36-2a634ebab021 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 09:04:01.119060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:01.119060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:01.119060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:01.119060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:01.119060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:01.119060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:01.119060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:01.119060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:01.119060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:01.119060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:01.119060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:01.119060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:01.119060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:01.119060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:01.119060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:01.120292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:01.120292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:01.120292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:01.120292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:01.120292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:01.120292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:01.120292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:01.120292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:01.120292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:01.120292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:01.120292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:01.120292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:01.120292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:01.120292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:01.120292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:01.120292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:01.120292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:01.120292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:01.121243 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:01.121243 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:01.121243 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:01.121243 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:01.121243 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:01.121243 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:01.121243 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:01.121243 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:01.121243 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 09:04:01.121243 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:01.121243 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 251/508] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:04:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:01.121872 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785f041-d74d-483c-9b36-2a634ebab021 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:01.135017 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d891c9-ca5e-4ab1-8ac4-a8f14e69e549 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:01.168891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d11cb-fa25-422d-a5ad-fc537589680d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:01.169562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d11cb-fa25-422d-a5ad-fc537589680d None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:01.169854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d11cb-fa25-422d-a5ad-fc537589680d None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:01.170135 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d11cb-fa25-422d-a5ad-fc537589680d None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:01.170868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d11cb-fa25-422d-a5ad-fc537589680d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:01.171211 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d11cb-fa25-422d-a5ad-fc537589680d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:01.172599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d11cb-fa25-422d-a5ad-fc537589680d None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:01.184808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 258/509] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:01.185608 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d11cb-fa25-422d-a5ad-fc537589680d None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:01.229344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785f041-d74d-483c-9b36-2a634ebab021 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:01.262161 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71128d34-652d-463d-8e9f-acf4ee638870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:01.262826 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71128d34-652d-463d-8e9f-acf4ee638870 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:01.263146 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71128d34-652d-463d-8e9f-acf4ee638870 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:01.263433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71128d34-652d-463d-8e9f-acf4ee638870 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:01.264176 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71128d34-652d-463d-8e9f-acf4ee638870 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:01.264490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71128d34-652d-463d-8e9f-acf4ee638870 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:01.265873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71128d34-652d-463d-8e9f-acf4ee638870 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:01.277287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 252/510] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:01.278615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71128d34-652d-463d-8e9f-acf4ee638870 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:01.282306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d11cb-fa25-422d-a5ad-fc537589680d None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:01.296334 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:01.316485 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c72cf2-9240-452f-9519-6735d43f1391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:01.317161 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c72cf2-9240-452f-9519-6735d43f1391 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:01.317444 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c72cf2-9240-452f-9519-6735d43f1391 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:01.317730 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c72cf2-9240-452f-9519-6735d43f1391 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:01.318478 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c72cf2-9240-452f-9519-6735d43f1391 None admin] RBAC: Adding query filter params (name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:01.318786 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c72cf2-9240-452f-9519-6735d43f1391 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:01.320237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c72cf2-9240-452f-9519-6735d43f1391 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:01.328759 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 259/511] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jun 23 09:04:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:01.329545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c72cf2-9240-452f-9519-6735d43f1391 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:01.347819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:01.388990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71128d34-652d-463d-8e9f-acf4ee638870 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:01.424433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001314ed-af10-4fa2-903c-bdde98314c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:01.425097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001314ed-af10-4fa2-903c-bdde98314c22 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:01.425377 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001314ed-af10-4fa2-903c-bdde98314c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:01.425653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001314ed-af10-4fa2-903c-bdde98314c22 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:01.426374 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001314ed-af10-4fa2-903c-bdde98314c22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:01.426678 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001314ed-af10-4fa2-903c-bdde98314c22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:01.428092 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001314ed-af10-4fa2-903c-bdde98314c22 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:01.428484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c72cf2-9240-452f-9519-6735d43f1391 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:01.439490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 253/512] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:01.440270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001314ed-af10-4fa2-903c-bdde98314c22 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:01.463024 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d126fe-8967-46f4-9c19-ef4bb818d3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:01.463702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d126fe-8967-46f4-9c19-ef4bb818d3ed None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:01.463993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d126fe-8967-46f4-9c19-ef4bb818d3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:01.464327 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d126fe-8967-46f4-9c19-ef4bb818d3ed None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:01.465706 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d126fe-8967-46f4-9c19-ef4bb818d3ed None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:01.466023 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d126fe-8967-46f4-9c19-ef4bb818d3ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:01.467483 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d126fe-8967-46f4-9c19-ef4bb818d3ed None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:01.484260 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 260/513] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jun 23 09:04:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:01.484990 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d126fe-8967-46f4-9c19-ef4bb818d3ed None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:01.517957 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:01.540726 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001314ed-af10-4fa2-903c-bdde98314c22 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:01.558421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:01.576011 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ceccd-a245-4ac6-b3c0-9e2746f062fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:01.576810 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ceccd-a245-4ac6-b3c0-9e2746f062fc None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:01.577098 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ceccd-a245-4ac6-b3c0-9e2746f062fc None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:01.577378 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ceccd-a245-4ac6-b3c0-9e2746f062fc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:01.585074 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ceccd-a245-4ac6-b3c0-9e2746f062fc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:01.586621 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ceccd-a245-4ac6-b3c0-9e2746f062fc None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:01.587454 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d126fe-8967-46f4-9c19-ef4bb818d3ed None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:01.601118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-3d0ceccd-a245-4ac6-b3c0-9e2746f062fc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 09:04:01.601118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:01.601118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:01.601118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:01.601118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:01.601118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:01.601118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:01.601118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:01.601118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:01.601118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:01.601118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:01.601118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:01.601118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:01.601118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:01.601118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:01.601118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:01.602157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:01.602157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:01.602157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:01.602157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:01.602157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:01.602157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:01.602157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:01.602157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:01.602157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:01.602157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:01.602157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:01.602157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:01.602157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:01.602157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:01.602157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:01.602157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:01.602157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:01.602157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:01.603139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:01.603139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:01.603139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:01.603139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:01.603139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:01.603139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:01.603139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:01.603139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:01.603139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 09:04:01.603139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:01.603139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 254/514] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:04:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:01.603139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ceccd-a245-4ac6-b3c0-9e2746f062fc None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:01.627081 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef03fb1-6506-4b17-aee7-649d372915a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:01.627890 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef03fb1-6506-4b17-aee7-649d372915a5 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:01.628180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef03fb1-6506-4b17-aee7-649d372915a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:01.628464 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef03fb1-6506-4b17-aee7-649d372915a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:01.635213 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef03fb1-6506-4b17-aee7-649d372915a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:01.636672 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef03fb1-6506-4b17-aee7-649d372915a5 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:01.651255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-bef03fb1-6506-4b17-aee7-649d372915a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:01.651255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:01.651255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:01.651255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:01.651255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:01.651255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:01.651255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:01.651255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:01.651255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:01.651255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:01.651255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:01.651255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:01.651255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:01.651255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:01.651255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:01.652319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:01.652319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:01.652319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:01.652319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:01.652319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:01.652319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:01.652319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:01.652319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:01.652319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:01.652319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:01.652319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:01.652319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:01.652319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:01.652319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:01.652319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:01.652319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:01.652319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:01.653995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:01.653995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:01.653995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:01.653995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:01.653995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:01.653995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:01.653995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:01.653995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:01.653995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 261/515] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:01.653995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef03fb1-6506-4b17-aee7-649d372915a5 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:01.707640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ceccd-a245-4ac6-b3c0-9e2746f062fc None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:01.742032 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf051d35-3400-433b-abeb-a9c4a0781ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:01.742754 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf051d35-3400-433b-abeb-a9c4a0781ddf None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:01.743071 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf051d35-3400-433b-abeb-a9c4a0781ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:01.743352 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf051d35-3400-433b-abeb-a9c4a0781ddf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:01.750464 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:01.753654 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf051d35-3400-433b-abeb-a9c4a0781ddf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:01.755142 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf051d35-3400-433b-abeb-a9c4a0781ddf None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:01.757091 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef03fb1-6506-4b17-aee7-649d372915a5 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:01.772857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-bf051d35-3400-433b-abeb-a9c4a0781ddf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:01.772857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:01.772857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:01.772857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:01.772857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:01.772857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:01.772857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:01.772857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:01.772857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:01.772857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:01.772857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:01.772857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:01.772857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:01.772857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:01.772857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:01.773952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:01.773952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:01.773952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:01.773952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:01.773952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:01.773952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:01.773952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:01.773952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:01.773952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:01.773952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:01.773952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:01.773952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:01.773952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:01.773952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:01.773952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:01.773952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:01.773952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:01.775363 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:01.775363 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:01.775363 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:01.775363 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:01.775363 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 255/516] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:01.775363 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf051d35-3400-433b-abeb-a9c4a0781ddf None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:01.792987 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478142a1-a0c1-4372-98c2-cb22e9a89886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:01.794168 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478142a1-a0c1-4372-98c2-cb22e9a89886 None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:01.794455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478142a1-a0c1-4372-98c2-cb22e9a89886 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:01.794734 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478142a1-a0c1-4372-98c2-cb22e9a89886 None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:01.795476 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478142a1-a0c1-4372-98c2-cb22e9a89886 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:01.796970 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478142a1-a0c1-4372-98c2-cb22e9a89886 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:01.808059 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:01.874879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf051d35-3400-433b-abeb-a9c4a0781ddf None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:01.907831 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00d2a7-e7c2-4352-b7a9-804af27894e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:01.908484 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00d2a7-e7c2-4352-b7a9-804af27894e4 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:01.908767 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00d2a7-e7c2-4352-b7a9-804af27894e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:01.909041 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00d2a7-e7c2-4352-b7a9-804af27894e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:01.926124 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 262/517] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jun 23 09:04:01 2022] POST /identity/v3/users => generated 247 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:04:01.926124 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478142a1-a0c1-4372-98c2-cb22e9a89886 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:01.984696 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00d2a7-e7c2-4352-b7a9-804af27894e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:01.986298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00d2a7-e7c2-4352-b7a9-804af27894e4 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:02.019159 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:02.046883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478142a1-a0c1-4372-98c2-cb22e9a89886 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:02.091105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae4ce4-b28f-46d6-80be-c4f26419264e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:02.091789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae4ce4-b28f-46d6-80be-c4f26419264e None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:02.092074 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae4ce4-b28f-46d6-80be-c4f26419264e None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:02.092350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae4ce4-b28f-46d6-80be-c4f26419264e None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:02.093194 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae4ce4-b28f-46d6-80be-c4f26419264e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:02.093590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae4ce4-b28f-46d6-80be-c4f26419264e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:02.096148 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae4ce4-b28f-46d6-80be-c4f26419264e None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:02.098160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 256/518] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:04:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:04:02.099045 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00d2a7-e7c2-4352-b7a9-804af27894e4 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:02.116394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 263/519] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jun 23 09:04:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:02.117010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae4ce4-b28f-46d6-80be-c4f26419264e None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:02.128456 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:02.221016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00d2a7-e7c2-4352-b7a9-804af27894e4 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:02.232909 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae4ce4-b28f-46d6-80be-c4f26419264e None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:02.251758 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:02.258329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ba1c5-3a52-4857-b8c8-c1024c2d6206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:02.259089 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ba1c5-3a52-4857-b8c8-c1024c2d6206 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:02.259376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ba1c5-3a52-4857-b8c8-c1024c2d6206 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:02.259653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ba1c5-3a52-4857-b8c8-c1024c2d6206 None admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:02.260445 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ba1c5-3a52-4857-b8c8-c1024c2d6206 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:02.260757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ba1c5-3a52-4857-b8c8-c1024c2d6206 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:02.262862 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ba1c5-3a52-4857-b8c8-c1024c2d6206 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:02.272437 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfdc003-ccaa-4b5d-b560-9d237fd1c373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:02.273052 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfdc003-ccaa-4b5d-b560-9d237fd1c373 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:02.273820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfdc003-ccaa-4b5d-b560-9d237fd1c373 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:02.274112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfdc003-ccaa-4b5d-b560-9d237fd1c373 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:02.274851 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfdc003-ccaa-4b5d-b560-9d237fd1c373 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:02.275200 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfdc003-ccaa-4b5d-b560-9d237fd1c373 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:02.276583 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfdc003-ccaa-4b5d-b560-9d237fd1c373 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:02.285098 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 257/520] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:02.285098 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ba1c5-3a52-4857-b8c8-c1024c2d6206 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:02.293637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 264/521] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:02.294773 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfdc003-ccaa-4b5d-b560-9d237fd1c373 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:02.365579 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:02.394646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ba1c5-3a52-4857-b8c8-c1024c2d6206 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:02.403363 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfdc003-ccaa-4b5d-b560-9d237fd1c373 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:02.435595 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91407e11-2513-4be1-b677-131cb4cf9fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:02.436398 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91407e11-2513-4be1-b677-131cb4cf9fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:02.436683 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91407e11-2513-4be1-b677-131cb4cf9fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:02.436953 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91407e11-2513-4be1-b677-131cb4cf9fe5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:02.438732 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31499213-e67f-4e0a-8a8d-04a5df953323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:02.439420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31499213-e67f-4e0a-8a8d-04a5df953323 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:02.439711 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31499213-e67f-4e0a-8a8d-04a5df953323 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:02.439991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31499213-e67f-4e0a-8a8d-04a5df953323 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:02.450662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91407e11-2513-4be1-b677-131cb4cf9fe5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:02.452397 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91407e11-2513-4be1-b677-131cb4cf9fe5 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:02.470269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-91407e11-2513-4be1-b677-131cb4cf9fe5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:02.470269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:02.470269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:02.470269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:02.470269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:02.470269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:02.470269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:02.470269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:02.470269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:02.470269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:02.470269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:02.470269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:02.470269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:02.470269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:02.470269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:02.471263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:02.471263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:02.471263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:02.471263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:02.471263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:02.471263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:02.471263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:02.471263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:02.471263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:02.471263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:02.471263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:02.471263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:02.471263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:02.471263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:02.471263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:02.471263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:02.471263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:02.472272 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:02.472272 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:02.472272 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:02.472272 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:02.472272 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 258/522] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:02.472690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91407e11-2513-4be1-b677-131cb4cf9fe5 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:02.517990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:02.519503 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31499213-e67f-4e0a-8a8d-04a5df953323 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:02.520975 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31499213-e67f-4e0a-8a8d-04a5df953323 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:02.582177 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91407e11-2513-4be1-b677-131cb4cf9fe5 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:02.595905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:02.620413 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7558f2-1a35-47b2-a899-7cca7d35f336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:02.621071 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7558f2-1a35-47b2-a899-7cca7d35f336 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:02.621413 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7558f2-1a35-47b2-a899-7cca7d35f336 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:02.621695 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7558f2-1a35-47b2-a899-7cca7d35f336 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:02.622821 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7558f2-1a35-47b2-a899-7cca7d35f336 None admin] RBAC: Adding query filter params (role.id=edf696ae763147cab68434524d1eff9c, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=ff025cb706d243778468ae4570374690) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:02.623234 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7558f2-1a35-47b2-a899-7cca7d35f336 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:02.624673 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7558f2-1a35-47b2-a899-7cca7d35f336 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:02.630354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 259/523] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:04:02 2022] GET /identity/v3/role_assignments?user.id=ff025cb706d243778468ae4570374690&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=edf696ae763147cab68434524d1eff9c => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:02.635154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 265/524] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:04:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:04:02.635154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31499213-e67f-4e0a-8a8d-04a5df953323 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:02.637346 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7558f2-1a35-47b2-a899-7cca7d35f336 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:02.769406 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7558f2-1a35-47b2-a899-7cca7d35f336 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:02.770046 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31499213-e67f-4e0a-8a8d-04a5df953323 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:02.781238 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:02.814527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0079d9-c8b6-4696-a1f1-ceb51669a1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:02.814752 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0079d9-c8b6-4696-a1f1-ceb51669a1fb None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:02.815067 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0079d9-c8b6-4696-a1f1-ceb51669a1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:02.815257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0079d9-c8b6-4696-a1f1-ceb51669a1fb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:02.822466 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bfbd7-32ff-4a32-b12d-4ee22903f955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:02.823186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bfbd7-32ff-4a32-b12d-4ee22903f955 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:02.823473 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bfbd7-32ff-4a32-b12d-4ee22903f955 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:02.823752 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bfbd7-32ff-4a32-b12d-4ee22903f955 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:02.824814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382bfbd7-32ff-4a32-b12d-4ee22903f955 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:02.825130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382bfbd7-32ff-4a32-b12d-4ee22903f955 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:02.826647 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382bfbd7-32ff-4a32-b12d-4ee22903f955 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:02.828956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0079d9-c8b6-4696-a1f1-ceb51669a1fb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:02.830679 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0079d9-c8b6-4696-a1f1-ceb51669a1fb None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:02.840742 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 266/525] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:02.841652 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bfbd7-32ff-4a32-b12d-4ee22903f955 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:02.847508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-cf0079d9-c8b6-4696-a1f1-ceb51669a1fb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 09:04:02.847508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:02.847508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:02.847508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:02.847508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:02.847508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:02.847508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:02.847508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:02.847508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:02.847508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:02.847508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:02.847508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:02.847508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:02.847508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:02.847508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:02.847508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:02.848487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:02.848487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:02.848487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:02.848487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:02.848487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:02.848487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:02.848487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:02.848487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:02.848487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:02.848487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:02.848487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:02.848487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:02.848487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:02.848487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:02.848487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:02.848487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:02.848487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:02.848487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:02.849456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:02.849456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:02.849456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:02.849456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:02.849456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:02.849456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:02.849456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:02.849456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:02.849456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 09:04:02.849456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:02.856602 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 260/526] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:04:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:02.857709 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0079d9-c8b6-4696-a1f1-ceb51669a1fb None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:02.871112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:02.964662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0079d9-c8b6-4696-a1f1-ceb51669a1fb None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:02.969172 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bfbd7-32ff-4a32-b12d-4ee22903f955 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:03.004713 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca85483-cc1d-4327-988d-4c6ef2562eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:03.005768 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca85483-cc1d-4327-988d-4c6ef2562eaf None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:03.006229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca85483-cc1d-4327-988d-4c6ef2562eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:03.006659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca85483-cc1d-4327-988d-4c6ef2562eaf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:03.011484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787ab8b-ec91-4913-9431-f244117b6cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:03.012810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787ab8b-ec91-4913-9431-f244117b6cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:03.013101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787ab8b-ec91-4913-9431-f244117b6cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:03.013384 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787ab8b-ec91-4913-9431-f244117b6cd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:03.016074 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca85483-cc1d-4327-988d-4c6ef2562eaf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:03.017771 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca85483-cc1d-4327-988d-4c6ef2562eaf None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:03.024435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787ab8b-ec91-4913-9431-f244117b6cd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:03.025906 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787ab8b-ec91-4913-9431-f244117b6cd6 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:03.033641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-6ca85483-cc1d-4327-988d-4c6ef2562eaf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:03.033641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:03.033641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:03.033641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:03.033641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:03.033641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:03.033641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:03.033641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:03.033641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:03.033641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:03.033641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:03.033641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:03.033641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:03.033641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:03.033641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:03.034615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:03.034615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:03.034615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:03.034615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:03.034615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:03.034615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:03.034615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:03.034615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:03.034615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:03.034615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:03.034615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:03.034615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:03.034615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:03.034615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:03.034615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:03.034615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:03.034615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:03.035564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:03.035564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:03.035564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:03.035564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:03.035564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:03.035564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:03.035564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:03.035564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:03.037770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 261/527] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:03.038757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca85483-cc1d-4327-988d-4c6ef2562eaf None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:03.048891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-e787ab8b-ec91-4913-9431-f244117b6cd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:03.048891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:03.048891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:03.048891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:03.048891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:03.048891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:03.048891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:03.048891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:03.048891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:03.048891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:03.048891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:03.048891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:03.048891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:03.048891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:03.048891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:03.049848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:03.049848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:03.049848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:03.049848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:03.049848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:03.049848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:03.049848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:03.049848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:03.049848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:03.049848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:03.049848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:03.049848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:03.049848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:03.049848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:03.049848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:03.049848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:03.049848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:03.050765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:03.050765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:03.050765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:03.050765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:03.054893 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 267/528] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:03.055577 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787ab8b-ec91-4913-9431-f244117b6cd6 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:03.071754 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:03.155631 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca85483-cc1d-4327-988d-4c6ef2562eaf None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:03.157928 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:03.180380 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787ab8b-ec91-4913-9431-f244117b6cd6 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:03.208677 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d875102-5dff-4c1e-82ef-7ae18733cd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:03.209948 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d875102-5dff-4c1e-82ef-7ae18733cd46 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:03.210772 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d875102-5dff-4c1e-82ef-7ae18733cd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:03.211529 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d875102-5dff-4c1e-82ef-7ae18733cd46 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:03.212854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d875102-5dff-4c1e-82ef-7ae18733cd46 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:03.213754 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d875102-5dff-4c1e-82ef-7ae18733cd46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:03.216069 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d875102-5dff-4c1e-82ef-7ae18733cd46 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:03.218439 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f52c80-6d0d-42d1-865e-c742e8054da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:03.219316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f52c80-6d0d-42d1-865e-c742e8054da8 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:03.219677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f52c80-6d0d-42d1-865e-c742e8054da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:03.220029 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f52c80-6d0d-42d1-865e-c742e8054da8 None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:03.220897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f52c80-6d0d-42d1-865e-c742e8054da8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:03.221280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f52c80-6d0d-42d1-865e-c742e8054da8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:03.222795 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f52c80-6d0d-42d1-865e-c742e8054da8 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:03.247343 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 268/529] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:03.248202 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f52c80-6d0d-42d1-865e-c742e8054da8 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:03.266261 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 262/530] 10.222.0.52 () {56 vars in 1139 bytes} [Thu Jun 23 09:04:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:03.267558 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d875102-5dff-4c1e-82ef-7ae18733cd46 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:03.363450 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f52c80-6d0d-42d1-865e-c742e8054da8 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:03.383150 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:03.399007 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852d1bb-c0f3-43ba-9530-9821bac93a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:03.399943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852d1bb-c0f3-43ba-9530-9821bac93a69 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:03.400330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852d1bb-c0f3-43ba-9530-9821bac93a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:03.400663 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852d1bb-c0f3-43ba-9530-9821bac93a69 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:03.401609 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852d1bb-c0f3-43ba-9530-9821bac93a69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:03.401794 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852d1bb-c0f3-43ba-9530-9821bac93a69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:03.404125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852d1bb-c0f3-43ba-9530-9821bac93a69 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:03.406214 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d875102-5dff-4c1e-82ef-7ae18733cd46 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:03.413765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:03.417355 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 269/531] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:03.418114 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852d1bb-c0f3-43ba-9530-9821bac93a69 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:03.455127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69be89f-271e-4138-896e-5b55d2c2152e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:03.456107 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69be89f-271e-4138-896e-5b55d2c2152e None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:03.456457 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69be89f-271e-4138-896e-5b55d2c2152e None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:03.456744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69be89f-271e-4138-896e-5b55d2c2152e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:03.457589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69be89f-271e-4138-896e-5b55d2c2152e None admin] RBAC: Adding query filter params (role.id=edf696ae763147cab68434524d1eff9c, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=36a6a3190dad4948a505bbe4c7095699) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:03.458064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69be89f-271e-4138-896e-5b55d2c2152e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:03.460392 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69be89f-271e-4138-896e-5b55d2c2152e None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:03.467643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 263/532] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:04:03 2022] GET /identity/v3/role_assignments?user.id=36a6a3190dad4948a505bbe4c7095699&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=edf696ae763147cab68434524d1eff9c => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:03.468364 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69be89f-271e-4138-896e-5b55d2c2152e None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:03.544548 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852d1bb-c0f3-43ba-9530-9821bac93a69 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:03.581149 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eea201-4fde-4c72-8235-5d1355166073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:03.581588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eea201-4fde-4c72-8235-5d1355166073 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:03.581882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eea201-4fde-4c72-8235-5d1355166073 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:03.582161 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eea201-4fde-4c72-8235-5d1355166073 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:03.618805 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69be89f-271e-4138-896e-5b55d2c2152e None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:03.664138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:03.667246 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eea201-4fde-4c72-8235-5d1355166073 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:03.667801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b490f1-485c-4e73-8745-d39efae9209d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:03.668805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eea201-4fde-4c72-8235-5d1355166073 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:03.669039 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b490f1-485c-4e73-8745-d39efae9209d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:03.669475 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b490f1-485c-4e73-8745-d39efae9209d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:03.669906 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b490f1-485c-4e73-8745-d39efae9209d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:03.682135 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b490f1-485c-4e73-8745-d39efae9209d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:03.686393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b490f1-485c-4e73-8745-d39efae9209d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:03.720370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-94b490f1-485c-4e73-8745-d39efae9209d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:03.720370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:03.720370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:03.720370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:03.720370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:03.720370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:03.720370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:03.720370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:03.720370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:03.720370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:03.720370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:03.720370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:03.720370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:03.720370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:03.720370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:03.721527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:03.721527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:03.721527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:03.721527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:03.721527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:03.721527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:03.721527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:03.721527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:03.721527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:03.721527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:03.721527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:03.721527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:03.721527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:03.721527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:03.721527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:03.721527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:03.721527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:03.722532 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:03.722532 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:03.722532 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:03.722532 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:03.722532 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:03.722532 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:03.722532 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:03.722532 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:03.724446 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 264/533] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:03.730380 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b490f1-485c-4e73-8745-d39efae9209d None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:03.747804 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:03.788281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 270/534] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:04:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:04:03.795382 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eea201-4fde-4c72-8235-5d1355166073 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:03.902557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b490f1-485c-4e73-8745-d39efae9209d None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:03.951014 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a5d4e-4af7-4a8b-8310-72d8d561931f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:03.951014 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a5d4e-4af7-4a8b-8310-72d8d561931f None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:03.951014 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a5d4e-4af7-4a8b-8310-72d8d561931f None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:03.951014 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a5d4e-4af7-4a8b-8310-72d8d561931f None admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:03.951861 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a5d4e-4af7-4a8b-8310-72d8d561931f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:03.952277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a5d4e-4af7-4a8b-8310-72d8d561931f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:03.953973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a5d4e-4af7-4a8b-8310-72d8d561931f None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:03.968019 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 265/535] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:03.971402 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:03.973850 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a5d4e-4af7-4a8b-8310-72d8d561931f None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:03.978378 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eea201-4fde-4c72-8235-5d1355166073 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:04.030743 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f559acf-8b6c-4e3a-8062-0e31001d54f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:04.031612 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f559acf-8b6c-4e3a-8062-0e31001d54f4 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:04.031902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f559acf-8b6c-4e3a-8062-0e31001d54f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:04.032184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f559acf-8b6c-4e3a-8062-0e31001d54f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:04.044336 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f559acf-8b6c-4e3a-8062-0e31001d54f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:04.046555 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f559acf-8b6c-4e3a-8062-0e31001d54f4 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:04.066635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-1f559acf-8b6c-4e3a-8062-0e31001d54f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:04.066635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:04.066635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:04.066635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:04.066635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:04.066635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:04.066635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:04.066635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:04.066635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:04.066635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:04.066635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:04.066635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:04.066635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:04.066635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:04.066635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:04.077083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:04.077083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:04.077083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:04.077083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:04.077083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:04.077083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:04.077083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:04.077083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:04.077083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:04.077083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:04.077083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:04.077083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:04.077083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:04.077083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:04.077083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:04.077083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:04.077083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:04.078000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:04.078000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:04.078000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:04.078000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:04.078000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 271/536] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:04.078000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f559acf-8b6c-4e3a-8062-0e31001d54f4 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:04.182073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:04.222255 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a5d4e-4af7-4a8b-8310-72d8d561931f None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:04.260528 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e1a52-0226-4ae3-8e21-2a499c590f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:04.261198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e1a52-0226-4ae3-8e21-2a499c590f6a None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:04.261542 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e1a52-0226-4ae3-8e21-2a499c590f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:04.262417 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e1a52-0226-4ae3-8e21-2a499c590f6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:04.263363 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e1a52-0226-4ae3-8e21-2a499c590f6a None admin] RBAC: Adding query filter params (role.id=edf696ae763147cab68434524d1eff9c, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=c21a758d0e2645318c89a4fe8805df0a) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:04.263685 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e1a52-0226-4ae3-8e21-2a499c590f6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:04.265403 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e1a52-0226-4ae3-8e21-2a499c590f6a None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:04.278942 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 266/537] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:04:03 2022] GET /identity/v3/role_assignments?user.id=c21a758d0e2645318c89a4fe8805df0a&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=edf696ae763147cab68434524d1eff9c => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:04.278942 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686d672-d4bf-491a-a231-92f67c9753f5 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:04.278942 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686d672-d4bf-491a-a231-92f67c9753f5 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:04.278942 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686d672-d4bf-491a-a231-92f67c9753f5 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:04.279396 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 267/538] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:04.281250 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0db0b-adb3-48dc-9836-22b9943bc30a None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:04.281548 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0db0b-adb3-48dc-9836-22b9943bc30a None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:04.281835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0db0b-adb3-48dc-9836-22b9943bc30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:04.283320 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f559acf-8b6c-4e3a-8062-0e31001d54f4 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:04.306814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-43b0db0b-adb3-48dc-9836-22b9943bc30a None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:04.308090 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-43b0db0b-adb3-48dc-9836-22b9943bc30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:04.355713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db69aaa-dbf1-4e9c-ac6c-4bbe9af95b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:04.356412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db69aaa-dbf1-4e9c-ac6c-4bbe9af95b8b None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:04.356704 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db69aaa-dbf1-4e9c-ac6c-4bbe9af95b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:04.356986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db69aaa-dbf1-4e9c-ac6c-4bbe9af95b8b None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:04.357736 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db69aaa-dbf1-4e9c-ac6c-4bbe9af95b8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:04.358049 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db69aaa-dbf1-4e9c-ac6c-4bbe9af95b8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:04.359504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db69aaa-dbf1-4e9c-ac6c-4bbe9af95b8b None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:04.394559 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 272/539] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:04.395636 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db69aaa-dbf1-4e9c-ac6c-4bbe9af95b8b None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:04.416172 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:04.475012 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 268/540] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 09:04:04.494499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0db0b-adb3-48dc-9836-22b9943bc30a None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:04.556360 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db69aaa-dbf1-4e9c-ac6c-4bbe9af95b8b None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:04.598379 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdd151-7565-43c2-a6aa-73f056858f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:04.599579 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdd151-7565-43c2-a6aa-73f056858f97 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:04.600033 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdd151-7565-43c2-a6aa-73f056858f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:04.600472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdd151-7565-43c2-a6aa-73f056858f97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:04.608673 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdd151-7565-43c2-a6aa-73f056858f97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:04.610215 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:04.611891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdd151-7565-43c2-a6aa-73f056858f97 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:04.627208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-1cfdd151-7565-43c2-a6aa-73f056858f97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:04.627208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:04.627208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:04.627208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:04.627208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:04.627208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:04.627208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:04.627208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:04.627208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:04.627208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:04.627208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:04.627208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:04.627208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:04.627208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:04.627208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:04.628455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:04.628455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:04.628455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:04.628455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:04.628455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:04.628455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:04.628455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:04.628455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:04.628455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:04.628455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:04.628455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:04.628455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:04.628455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:04.628455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:04.628455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:04.628455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:04.628455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:04.629617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:04.629617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:04.629617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:04.629617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:04.629617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:04.629617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:04.629617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:04.629617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:04.631500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 273/541] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:04.638472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdd151-7565-43c2-a6aa-73f056858f97 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:04.691316 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0db0b-adb3-48dc-9836-22b9943bc30a None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:04.757041 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41404aaf-c0be-4b19-8989-06bb33a057a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:04.757769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41404aaf-c0be-4b19-8989-06bb33a057a5 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:04.758055 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41404aaf-c0be-4b19-8989-06bb33a057a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:04.758334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41404aaf-c0be-4b19-8989-06bb33a057a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:04.762788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:04.795830 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41404aaf-c0be-4b19-8989-06bb33a057a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:04.797118 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdd151-7565-43c2-a6aa-73f056858f97 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:04.807915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-41404aaf-c0be-4b19-8989-06bb33a057a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:04.807915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:04.807915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:04.807915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:04.807915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:04.807915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:04.807915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:04.807915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:04.807915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:04.807915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:04.807915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:04.807915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:04.807915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:04.807915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:04.807915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:04.807915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:04.808985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:04.808985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:04.808985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:04.808985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:04.808985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:04.808985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:04.808985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:04.808985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:04.808985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:04.808985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:04.808985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:04.808985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:04.808985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:04.808985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:04.808985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:04.808985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:04.809849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:04.809849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:04.809849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:04.809849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:04.809849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:04.809849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:04.809849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:04.809849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:04.809849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:04.809849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:04.809849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:04.809849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:04.813928 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 269/542] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:04:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:04.832809 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41404aaf-c0be-4b19-8989-06bb33a057a5 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:04.870327 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc0429-c4d3-41f4-a4c1-0d549d9559a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:04.871483 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc0429-c4d3-41f4-a4c1-0d549d9559a3 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:04.872612 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc0429-c4d3-41f4-a4c1-0d549d9559a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:04.873915 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc0429-c4d3-41f4-a4c1-0d549d9559a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:04.876592 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc0429-c4d3-41f4-a4c1-0d549d9559a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:04.877474 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc0429-c4d3-41f4-a4c1-0d549d9559a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:04.880899 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc0429-c4d3-41f4-a4c1-0d549d9559a3 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:04.897820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 274/543] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:04.903767 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc0429-c4d3-41f4-a4c1-0d549d9559a3 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:04.991974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41404aaf-c0be-4b19-8989-06bb33a057a5 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:05.010943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc0429-c4d3-41f4-a4c1-0d549d9559a3 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:05.037929 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849538f-012e-4978-8038-7ffeaaed09ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:05.039648 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849538f-012e-4978-8038-7ffeaaed09ac None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:05.040172 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849538f-012e-4978-8038-7ffeaaed09ac None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:05.040935 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849538f-012e-4978-8038-7ffeaaed09ac None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:05.041749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:05.043588 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849538f-012e-4978-8038-7ffeaaed09ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:05.043912 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849538f-012e-4978-8038-7ffeaaed09ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:05.045341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849538f-012e-4978-8038-7ffeaaed09ac None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:05.046944 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:05.055075 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6cdf1-0437-498d-a26c-1dba540ad13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:05.058099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6cdf1-0437-498d-a26c-1dba540ad13c None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:05.058099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6cdf1-0437-498d-a26c-1dba540ad13c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:05.058099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6cdf1-0437-498d-a26c-1dba540ad13c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:05.058099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6cdf1-0437-498d-a26c-1dba540ad13c None admin] RBAC: Adding query filter params (role.id=edf696ae763147cab68434524d1eff9c, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=daed809f511a43ed8d3f88eba10466b2) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:05.058099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6cdf1-0437-498d-a26c-1dba540ad13c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:05.059150 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 270/544] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jun 23 09:04:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:05.060170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6cdf1-0437-498d-a26c-1dba540ad13c None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:05.060486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01825d23-9e9a-4355-8c12-45f73a176d8e None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:05.060869 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01825d23-9e9a-4355-8c12-45f73a176d8e None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:05.061245 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01825d23-9e9a-4355-8c12-45f73a176d8e None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:05.062850 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 271/545] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:05.068490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58075ba-52b0-4de9-b6c3-6ec3238b4446 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:05.069173 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58075ba-52b0-4de9-b6c3-6ec3238b4446 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:05.069552 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58075ba-52b0-4de9-b6c3-6ec3238b4446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:05.070898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 275/546] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:04:04 2022] GET /identity/v3/role_assignments?user.id=daed809f511a43ed8d3f88eba10466b2&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=edf696ae763147cab68434524d1eff9c => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:05.071948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6cdf1-0437-498d-a26c-1dba540ad13c None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:05.111940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-f58075ba-52b0-4de9-b6c3-6ec3238b4446 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:05.113158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-f58075ba-52b0-4de9-b6c3-6ec3238b4446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:05.268176 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 272/547] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:04:05.280126 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6cdf1-0437-498d-a26c-1dba540ad13c None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:05.301662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58075ba-52b0-4de9-b6c3-6ec3238b4446 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:05.348243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db940eb9-e4e2-4961-8922-c3d8e2ac4cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:05.349181 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db940eb9-e4e2-4961-8922-c3d8e2ac4cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:05.350883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db940eb9-e4e2-4961-8922-c3d8e2ac4cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:05.351204 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db940eb9-e4e2-4961-8922-c3d8e2ac4cb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:05.375629 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db940eb9-e4e2-4961-8922-c3d8e2ac4cb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:05.377107 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db940eb9-e4e2-4961-8922-c3d8e2ac4cb5 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:05.410830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-db940eb9-e4e2-4961-8922-c3d8e2ac4cb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:05.410830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:05.410830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:05.410830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:05.410830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:05.410830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:05.410830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:05.410830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:05.410830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:05.410830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:05.410830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:05.410830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:05.410830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:05.410830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:05.410830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:05.411825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:05.411825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:05.411825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:05.411825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:05.411825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:05.411825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:05.411825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:05.411825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:05.411825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:05.411825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:05.411825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:05.411825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:05.411825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:05.411825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:05.411825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:05.411825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:05.411825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:05.413219 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:05.413219 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:05.413219 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:05.413219 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:05.413219 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 276/548] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:05.418165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db940eb9-e4e2-4961-8922-c3d8e2ac4cb5 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:05.463076 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58075ba-52b0-4de9-b6c3-6ec3238b4446 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:05.479763 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:05.496013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:05.544083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db940eb9-e4e2-4961-8922-c3d8e2ac4cb5 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:05.557687 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf210f-5921-451d-adc8-935a2cb45324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:05.558579 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf210f-5921-451d-adc8-935a2cb45324 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:05.558956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf210f-5921-451d-adc8-935a2cb45324 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:05.559223 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf210f-5921-451d-adc8-935a2cb45324 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:05.588253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf210f-5921-451d-adc8-935a2cb45324 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:05.592864 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6de36-b0ad-4cc5-8a3c-6e244a17c398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:05.593542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6de36-b0ad-4cc5-8a3c-6e244a17c398 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:05.593827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6de36-b0ad-4cc5-8a3c-6e244a17c398 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:05.594103 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6de36-b0ad-4cc5-8a3c-6e244a17c398 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:05.597398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6de36-b0ad-4cc5-8a3c-6e244a17c398 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:05.597715 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6de36-b0ad-4cc5-8a3c-6e244a17c398 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:05.601421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-78cf210f-5921-451d-adc8-935a2cb45324 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:05.601421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:05.601421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:05.601421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:05.601421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:05.601421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:05.601421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:05.601421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:05.601421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:05.601421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:05.601421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:05.601421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:05.601421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:05.601421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:05.601421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:05.601421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:05.603613 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:05.603613 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:05.603613 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:05.603613 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:05.603613 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:05.603613 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:05.603613 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:05.603613 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:05.603613 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:05.603613 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:05.603613 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:05.603613 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:05.603613 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:05.603613 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:05.603613 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:05.603613 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:05.606212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:05.606212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:05.606212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:05.606212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:05.606212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:05.606212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:05.606212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:05.606212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:05.606212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:05.606212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:05.606212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:05.606212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:05.607371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6de36-b0ad-4cc5-8a3c-6e244a17c398 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:05.607454 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 273/549] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:04:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:05.612289 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf210f-5921-451d-adc8-935a2cb45324 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:05.639032 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 277/550] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:05.654845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6de36-b0ad-4cc5-8a3c-6e244a17c398 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:05.765546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6de36-b0ad-4cc5-8a3c-6e244a17c398 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:05.784159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:05.801964 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ce151-54a5-4cac-ac6d-8dad924b3134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:05.802793 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ce151-54a5-4cac-ac6d-8dad924b3134 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:05.803106 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ce151-54a5-4cac-ac6d-8dad924b3134 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:05.803386 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ce151-54a5-4cac-ac6d-8dad924b3134 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:05.811916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ce151-54a5-4cac-ac6d-8dad924b3134 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:05.813391 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ce151-54a5-4cac-ac6d-8dad924b3134 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:05.830303 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf210f-5921-451d-adc8-935a2cb45324 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:05.830588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-046ce151-54a5-4cac-ac6d-8dad924b3134 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 09:04:05.830588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:05.830588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:05.830588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:05.830588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:05.830588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:05.830588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:05.830588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:05.830588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:05.830588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:05.830588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:05.830588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:05.830588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:05.830588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:05.830588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:05.830588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:05.832775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:05.832775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:05.832775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:05.832775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:05.832775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:05.832775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:05.832775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:05.832775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:05.832775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:05.832775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:05.832775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:05.832775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:05.832775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:05.832775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:05.832775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:05.832775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:05.832775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:05.832775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:05.835411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:05.835411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:05.835411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:05.835411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:05.835411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:05.835411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:05.835411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:05.835411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:05.835411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 09:04:05.835411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:05.836978 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 278/551] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:04:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:05.838940 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ce151-54a5-4cac-ac6d-8dad924b3134 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:05.895550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94df213-1863-4cac-880a-5f2845bccf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:05.895550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94df213-1863-4cac-880a-5f2845bccf6d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:05.895550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94df213-1863-4cac-880a-5f2845bccf6d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:05.895550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94df213-1863-4cac-880a-5f2845bccf6d None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:05.896357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94df213-1863-4cac-880a-5f2845bccf6d None admin] RBAC: Adding query filter params (name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:05.896357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94df213-1863-4cac-880a-5f2845bccf6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:05.896357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94df213-1863-4cac-880a-5f2845bccf6d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:05.906701 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 274/552] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jun 23 09:04:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:05.908358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714eac51-fd4b-4e89-aa40-76195c572d9e None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:05.908885 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714eac51-fd4b-4e89-aa40-76195c572d9e None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:05.909394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714eac51-fd4b-4e89-aa40-76195c572d9e None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:05.915095 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 275/553] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:05.916180 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714eac51-fd4b-4e89-aa40-76195c572d9e None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:05.947834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ce151-54a5-4cac-ac6d-8dad924b3134 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:05.955294 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:05.986099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e9688-4a1b-4b40-9213-0475c5c9b2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:05.986768 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e9688-4a1b-4b40-9213-0475c5c9b2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:05.987091 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e9688-4a1b-4b40-9213-0475c5c9b2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:05.987372 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e9688-4a1b-4b40-9213-0475c5c9b2d2 None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:05.988268 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e9688-4a1b-4b40-9213-0475c5c9b2d2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:05.988586 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e9688-4a1b-4b40-9213-0475c5c9b2d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:05.990034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e9688-4a1b-4b40-9213-0475c5c9b2d2 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:06.011126 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 279/554] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jun 23 09:04:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:06.012979 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891de490-f8e0-42aa-b0d3-5b9ca0fdd596 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:06.013272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891de490-f8e0-42aa-b0d3-5b9ca0fdd596 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:06.013608 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891de490-f8e0-42aa-b0d3-5b9ca0fdd596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:06.031438 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-891de490-f8e0-42aa-b0d3-5b9ca0fdd596 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:06.032624 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-891de490-f8e0-42aa-b0d3-5b9ca0fdd596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:06.037684 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714eac51-fd4b-4e89-aa40-76195c572d9e None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:06.054613 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:06.073855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfb0ef-f229-49a3-acbe-6335652950e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:06.075226 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfb0ef-f229-49a3-acbe-6335652950e9 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:06.075506 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfb0ef-f229-49a3-acbe-6335652950e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:06.075784 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfb0ef-f229-49a3-acbe-6335652950e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:06.088219 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfb0ef-f229-49a3-acbe-6335652950e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:06.090762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfb0ef-f229-49a3-acbe-6335652950e9 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:06.107183 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 280/555] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:04:06.108315 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891de490-f8e0-42aa-b0d3-5b9ca0fdd596 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:06.112282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-13dfb0ef-f229-49a3-acbe-6335652950e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:06.112282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:06.112282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:06.112282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:06.112282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:06.112282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:06.112282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:06.112282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:06.112282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:06.112282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:06.112282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:06.112282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:06.112282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:06.112282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:06.112282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:06.113390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:06.113390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:06.113390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:06.113390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:06.113390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:06.113390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:06.113390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:06.113390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:06.113390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:06.113390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:06.113390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:06.113390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:06.113390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:06.113390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:06.113390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:06.113390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:06.113390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:06.114512 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:06.114512 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:06.114512 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:06.114512 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:06.117699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 276/556] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:06.118515 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfb0ef-f229-49a3-acbe-6335652950e9 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:06.213261 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:06.226131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfb0ef-f229-49a3-acbe-6335652950e9 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:06.231705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891de490-f8e0-42aa-b0d3-5b9ca0fdd596 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:06.262645 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e96184-4f6f-496b-b1bd-4925fd92d462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:06.263464 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e96184-4f6f-496b-b1bd-4925fd92d462 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:06.263758 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e96184-4f6f-496b-b1bd-4925fd92d462 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:06.264038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e96184-4f6f-496b-b1bd-4925fd92d462 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:06.264445 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd7209-7e44-442d-9cf9-498cf76401e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:06.264784 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e96184-4f6f-496b-b1bd-4925fd92d462 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:06.265091 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e96184-4f6f-496b-b1bd-4925fd92d462 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:06.265599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd7209-7e44-442d-9cf9-498cf76401e5 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:06.266027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd7209-7e44-442d-9cf9-498cf76401e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:06.266456 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd7209-7e44-442d-9cf9-498cf76401e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:06.266605 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e96184-4f6f-496b-b1bd-4925fd92d462 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:06.284357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 277/557] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:06.285133 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e96184-4f6f-496b-b1bd-4925fd92d462 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:06.294553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:06.340231 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd7209-7e44-442d-9cf9-498cf76401e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:06.341917 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd7209-7e44-442d-9cf9-498cf76401e5 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:06.395765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e96184-4f6f-496b-b1bd-4925fd92d462 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:06.435164 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbcc91-29aa-47f2-afe7-a0dff1d78d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:06.435846 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbcc91-29aa-47f2-afe7-a0dff1d78d8f None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:06.436135 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbcc91-29aa-47f2-afe7-a0dff1d78d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:06.436417 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbcc91-29aa-47f2-afe7-a0dff1d78d8f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:06.447845 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbcc91-29aa-47f2-afe7-a0dff1d78d8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:06.448936 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 281/558] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:04:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:04:06.450559 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd7209-7e44-442d-9cf9-498cf76401e5 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:06.460962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-aafbcc91-29aa-47f2-afe7-a0dff1d78d8f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:06.460962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:06.460962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:06.460962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:06.460962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:06.460962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:06.460962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:06.460962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:06.460962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:06.460962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:06.460962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:06.460962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:06.460962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:06.460962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:06.460962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:06.460962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:06.462600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:06.462600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:06.462600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:06.462600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:06.462600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:06.462600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:06.462600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:06.462600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:06.462600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:06.462600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:06.462600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:06.462600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:06.462600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:06.462600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:06.462600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:06.462600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:06.463593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:06.463593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:06.463593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:06.463593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:06.463593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:06.463593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:06.463593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:06.463593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:06.463593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:06.463593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:06.463593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:06.463593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:06.463593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 278/559] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:04:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:06.463593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbcc91-29aa-47f2-afe7-a0dff1d78d8f None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:06.476111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:06.523315 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:06.563887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd7209-7e44-442d-9cf9-498cf76401e5 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:06.570861 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbcc91-29aa-47f2-afe7-a0dff1d78d8f None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:06.599206 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9cc294-f3f3-49d2-812f-38dcc57a2cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:06.600275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cc294-f3f3-49d2-812f-38dcc57a2cfa None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:06.600769 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cc294-f3f3-49d2-812f-38dcc57a2cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:06.601199 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cc294-f3f3-49d2-812f-38dcc57a2cfa None admin] PATH_INFO: `/v3/users/nova` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:06.607568 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ec161-22d0-4bb3-af05-ee333c43b9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:06.694429 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ec161-22d0-4bb3-af05-ee333c43b9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:06.694429 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ec161-22d0-4bb3-af05-ee333c43b9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:06.694429 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ec161-22d0-4bb3-af05-ee333c43b9c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:06.694718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cc294-f3f3-49d2-812f-38dcc57a2cfa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:06.694718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cc294-f3f3-49d2-812f-38dcc57a2cfa None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:06.694929 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ec161-22d0-4bb3-af05-ee333c43b9c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:06.694929 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ec161-22d0-4bb3-af05-ee333c43b9c3 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:06.695092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-2f9cc294-f3f3-49d2-812f-38dcc57a2cfa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 09:04:06.695092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:06.695092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:06.695092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:06.695092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:06.695092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:06.695092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:06.695092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:06.695092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:06.695092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:06.695092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:06.695092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:06.695092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:06.695092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:06.695092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:06.695092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:06.696228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:06.696228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:06.696228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:06.696228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:06.696228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:06.696228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:06.696228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:06.696228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:06.696228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:06.696228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:06.696228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:06.696228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:06.696228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:06.696228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:06.696228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:06.696228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:06.696228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:06.696228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:06.697376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:06.697376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:06.697376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:06.697376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:06.697376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:06.697376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:06.697376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:06.697376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:06.697376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 09:04:06.697376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:06.697376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 282/560] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jun 23 09:04:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:06.697376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9cc294-f3f3-49d2-812f-38dcc57a2cfa None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:06.698101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-ae3ec161-22d0-4bb3-af05-ee333c43b9c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:06.698101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:06.698101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:06.698101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:06.698101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:06.698101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:06.698101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:06.698101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:06.698101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:06.698101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:06.698101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:06.698101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:06.698101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:06.698101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:06.698101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:06.699013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:06.699013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:06.699013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:06.699013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:06.699013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:06.699013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:06.699013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:06.699013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:06.699013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:06.699013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:06.699013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:06.699013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:06.699013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:06.699013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:06.699013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:06.699013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:06.699013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:06.700178 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:06.700178 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:06.700178 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:06.700178 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:06.700178 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 279/561] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:06.700178 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ec161-22d0-4bb3-af05-ee333c43b9c3 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:06.714408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:06.736760 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9cc294-f3f3-49d2-812f-38dcc57a2cfa None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:06.756080 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ec161-22d0-4bb3-af05-ee333c43b9c3 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:06.776731 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:06.783476 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b8519-36db-4780-bc4c-132c5e61579d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:06.784162 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b8519-36db-4780-bc4c-132c5e61579d None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:06.784449 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b8519-36db-4780-bc4c-132c5e61579d None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:06.784725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b8519-36db-4780-bc4c-132c5e61579d None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:06.785484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b8519-36db-4780-bc4c-132c5e61579d None admin] RBAC: Adding query filter params (name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:06.785790 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b8519-36db-4780-bc4c-132c5e61579d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:06.787428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b8519-36db-4780-bc4c-132c5e61579d None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:06.808340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 283/562] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jun 23 09:04:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:06.808340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b8519-36db-4780-bc4c-132c5e61579d None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:06.813770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5660c41-f280-423c-9f18-34ff82ddb119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:06.814453 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5660c41-f280-423c-9f18-34ff82ddb119 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:06.814759 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5660c41-f280-423c-9f18-34ff82ddb119 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:06.815069 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5660c41-f280-423c-9f18-34ff82ddb119 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:06.816610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5660c41-f280-423c-9f18-34ff82ddb119 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:06.816610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5660c41-f280-423c-9f18-34ff82ddb119 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:06.817660 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5660c41-f280-423c-9f18-34ff82ddb119 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:06.837741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 280/563] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jun 23 09:04:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:06.845178 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7cb59-313c-4850-96b2-7d0b79fdcc84 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:06.845178 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7cb59-313c-4850-96b2-7d0b79fdcc84 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:06.845178 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7cb59-313c-4850-96b2-7d0b79fdcc84 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:06.845178 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 281/564] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:06.845178 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7cb59-313c-4850-96b2-7d0b79fdcc84 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:06.943946 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b8519-36db-4780-bc4c-132c5e61579d None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:06.983467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e1fc6-7d0f-4ec8-a995-435370497c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:06.984142 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e1fc6-7d0f-4ec8-a995-435370497c45 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:06.984432 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e1fc6-7d0f-4ec8-a995-435370497c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:06.984723 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e1fc6-7d0f-4ec8-a995-435370497c45 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:06.985447 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e1fc6-7d0f-4ec8-a995-435370497c45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:06.985757 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e1fc6-7d0f-4ec8-a995-435370497c45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:06.987274 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e1fc6-7d0f-4ec8-a995-435370497c45 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:06.993595 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7cb59-313c-4850-96b2-7d0b79fdcc84 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:07.004792 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 284/565] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:07.005748 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e1fc6-7d0f-4ec8-a995-435370497c45 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:07.013374 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:07.040938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877169b5-e0a7-40bc-ab2f-e685a749d880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:07.041601 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877169b5-e0a7-40bc-ab2f-e685a749d880 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:07.041816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877169b5-e0a7-40bc-ab2f-e685a749d880 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:07.042132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877169b5-e0a7-40bc-ab2f-e685a749d880 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:07.052402 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:07.054433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877169b5-e0a7-40bc-ab2f-e685a749d880 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:07.055995 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877169b5-e0a7-40bc-ab2f-e685a749d880 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:07.076739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-877169b5-e0a7-40bc-ab2f-e685a749d880 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:07.076739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:07.076739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:07.076739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:07.076739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:07.076739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:07.076739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:07.076739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:07.076739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:07.076739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:07.076739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:07.076739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:07.076739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:07.076739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:07.076739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:07.077720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:07.077720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:07.077720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:07.077720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:07.077720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:07.077720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:07.077720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:07.077720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:07.077720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:07.077720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:07.077720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:07.077720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:07.077720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:07.077720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:07.077720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:07.077720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:07.077720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:07.078661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:07.078661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:07.078661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:07.078661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:07.078661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 282/566] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:07.084091 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61c3b0-0634-464f-8ff7-ad8625d60666 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:07.084385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61c3b0-0634-464f-8ff7-ad8625d60666 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:07.084667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61c3b0-0634-464f-8ff7-ad8625d60666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:07.103377 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-ea61c3b0-0634-464f-8ff7-ad8625d60666 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:07.104936 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ea61c3b0-0634-464f-8ff7-ad8625d60666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:07.126832 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e1fc6-7d0f-4ec8-a995-435370497c45 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:07.189361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6385276-4e25-49e2-98c7-f8e7b0c911d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:07.190181 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6385276-4e25-49e2-98c7-f8e7b0c911d7 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:07.190466 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6385276-4e25-49e2-98c7-f8e7b0c911d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:07.190742 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6385276-4e25-49e2-98c7-f8e7b0c911d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:07.201976 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6385276-4e25-49e2-98c7-f8e7b0c911d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:07.203709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6385276-4e25-49e2-98c7-f8e7b0c911d7 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:07.207868 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 283/567] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:04:07.209187 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea61c3b0-0634-464f-8ff7-ad8625d60666 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:07.227883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-b6385276-4e25-49e2-98c7-f8e7b0c911d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:07.227883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:07.227883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:07.227883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:07.227883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:07.227883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:07.227883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:07.227883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:07.227883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:07.227883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:07.227883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:07.227883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:07.227883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:07.227883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:07.227883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:07.255101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:07.255101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:07.255101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:07.255101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:07.255101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:07.255101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:07.255101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:07.255101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:07.255101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:07.255101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:07.255101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:07.255101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:07.255101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:07.255101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:07.255101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:07.255101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:07.255101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:07.256296 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:07.256296 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:07.256296 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:07.256296 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:07.256296 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 285/568] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:07.256296 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6385276-4e25-49e2-98c7-f8e7b0c911d7 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:07.385088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:07.393085 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:07.395389 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6385276-4e25-49e2-98c7-f8e7b0c911d7 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:07.403253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea61c3b0-0634-464f-8ff7-ad8625d60666 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:07.427502 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7f871-6938-429a-83b4-a9c486ba96e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:07.428219 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7f871-6938-429a-83b4-a9c486ba96e8 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:07.428543 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac8a4c-ba15-481f-b3de-5c696554b1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:07.428681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7f871-6938-429a-83b4-a9c486ba96e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:07.428961 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7f871-6938-429a-83b4-a9c486ba96e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:07.429366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac8a4c-ba15-481f-b3de-5c696554b1cb None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:07.429653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac8a4c-ba15-481f-b3de-5c696554b1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:07.429853 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7f871-6938-429a-83b4-a9c486ba96e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:07.429947 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac8a4c-ba15-481f-b3de-5c696554b1cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:07.430537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7f871-6938-429a-83b4-a9c486ba96e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:07.431947 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7f871-6938-429a-83b4-a9c486ba96e8 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:07.437671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac8a4c-ba15-481f-b3de-5c696554b1cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:07.439620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac8a4c-ba15-481f-b3de-5c696554b1cb None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:07.443506 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 286/569] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:07.444285 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7f871-6938-429a-83b4-a9c486ba96e8 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:07.454278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-e7ac8a4c-ba15-481f-b3de-5c696554b1cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:07.454278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:07.454278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:07.454278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:07.454278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:07.454278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:07.454278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:07.454278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:07.454278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:07.454278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:07.454278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:07.454278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:07.454278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:07.454278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:07.454278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:07.455310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:07.455310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:07.455310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:07.455310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:07.455310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:07.455310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:07.455310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:07.455310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:07.455310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:07.455310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:07.455310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:07.455310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:07.455310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:07.455310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:07.455310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:07.455310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:07.455310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:07.456267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:07.456267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:07.456267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:07.456267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:07.456267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:07.456267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:07.456267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:07.456267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:07.456267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 284/570] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:07.456267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac8a4c-ba15-481f-b3de-5c696554b1cb None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:07.560206 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac8a4c-ba15-481f-b3de-5c696554b1cb None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:07.562109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7f871-6938-429a-83b4-a9c486ba96e8 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:07.596004 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbd3aa-6c94-459c-845c-3b03bf9264b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:07.596373 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769c80f-2dc8-4b60-99eb-aff3cdcf6604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:07.596797 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbd3aa-6c94-459c-845c-3b03bf9264b1 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:07.597033 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769c80f-2dc8-4b60-99eb-aff3cdcf6604 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:07.597132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbd3aa-6c94-459c-845c-3b03bf9264b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:07.597319 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769c80f-2dc8-4b60-99eb-aff3cdcf6604 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:07.597414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbd3aa-6c94-459c-845c-3b03bf9264b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:07.597597 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769c80f-2dc8-4b60-99eb-aff3cdcf6604 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:07.598319 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769c80f-2dc8-4b60-99eb-aff3cdcf6604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:07.598626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769c80f-2dc8-4b60-99eb-aff3cdcf6604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:07.600046 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769c80f-2dc8-4b60-99eb-aff3cdcf6604 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:07.607290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dbd3aa-6c94-459c-845c-3b03bf9264b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:07.623139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 285/571] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:07.623330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-93dbd3aa-6c94-459c-845c-3b03bf9264b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:07.623330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:07.623330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:07.623330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:07.623330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:07.623330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:07.623330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:07.623330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:07.623330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:07.623330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:07.623330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:07.623330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:07.623330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:07.623330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:07.623330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:07.623330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:07.624587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:07.624587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:07.624587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:07.624587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:07.624587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:07.624587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:07.624587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:07.624587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:07.624587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:07.624587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:07.624587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:07.624587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:07.624587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:07.624587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:07.624587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:07.624587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:07.625709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:07.625709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:07.625709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:07.625709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:07.625709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:07.625709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:07.625709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:07.625709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:07.625709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:07.625709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:07.625709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:07.625709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:07.626558 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd0513-cd04-430b-9089-4c634696f89c None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:07.626558 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd0513-cd04-430b-9089-4c634696f89c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:07.626558 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd0513-cd04-430b-9089-4c634696f89c None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:07.626558 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 286/572] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:07.626558 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd0513-cd04-430b-9089-4c634696f89c None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:07.626979 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 287/573] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:04:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:07.626979 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbd3aa-6c94-459c-845c-3b03bf9264b1 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:07.640215 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:07.643811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:07.727164 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd0513-cd04-430b-9089-4c634696f89c None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:07.732569 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbd3aa-6c94-459c-845c-3b03bf9264b1 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:07.762335 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e465cf5-8da5-400a-8e7c-1a0c2f639f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:07.763170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e465cf5-8da5-400a-8e7c-1a0c2f639f00 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:07.763462 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e465cf5-8da5-400a-8e7c-1a0c2f639f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:07.763737 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e465cf5-8da5-400a-8e7c-1a0c2f639f00 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:07.767599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21ec70-d842-4b7c-a33f-4bc2471cb14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:07.768263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21ec70-d842-4b7c-a33f-4bc2471cb14a None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:07.768553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21ec70-d842-4b7c-a33f-4bc2471cb14a None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:07.768834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21ec70-d842-4b7c-a33f-4bc2471cb14a None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:07.769627 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21ec70-d842-4b7c-a33f-4bc2471cb14a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:07.769943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21ec70-d842-4b7c-a33f-4bc2471cb14a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:07.770789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e465cf5-8da5-400a-8e7c-1a0c2f639f00 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:07.771422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21ec70-d842-4b7c-a33f-4bc2471cb14a None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:07.772284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e465cf5-8da5-400a-8e7c-1a0c2f639f00 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:07.783492 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 288/574] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:07.784118 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21ec70-d842-4b7c-a33f-4bc2471cb14a None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:07.786583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-1e465cf5-8da5-400a-8e7c-1a0c2f639f00 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 09:04:07.786583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:07.786583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:07.786583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:07.786583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:07.786583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:07.786583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:07.786583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:07.786583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:07.786583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:07.786583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:07.786583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:07.786583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:07.786583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:07.786583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:07.786583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:07.787599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:07.787599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:07.787599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:07.787599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:07.787599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:07.787599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:07.787599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:07.787599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:07.787599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:07.787599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:07.787599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:07.787599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:07.787599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:07.787599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:07.787599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:07.787599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:07.787599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:07.787599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:07.788611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:07.788611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:07.788611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:07.788611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:07.788611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:07.788611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:07.788611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:07.788611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:07.788611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 09:04:07.788611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:07.788611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 287/575] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:04:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:07.789249 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021791c7-8549-4910-8b0d-36285b95e6f7 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:07.790168 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021791c7-8549-4910-8b0d-36285b95e6f7 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:07.790168 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021791c7-8549-4910-8b0d-36285b95e6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:07.806407 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-021791c7-8549-4910-8b0d-36285b95e6f7 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:07.807632 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-021791c7-8549-4910-8b0d-36285b95e6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:07.859830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:07.890063 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 288/576] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:04:07.890272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21ec70-d842-4b7c-a33f-4bc2471cb14a None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:07.890955 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021791c7-8549-4910-8b0d-36285b95e6f7 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:07.903886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:07.935234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2faad8-f222-486b-91fb-4c0f16417e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:07.935923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2faad8-f222-486b-91fb-4c0f16417e58 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:07.936209 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2faad8-f222-486b-91fb-4c0f16417e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:07.936485 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2faad8-f222-486b-91fb-4c0f16417e58 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:07.937242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2faad8-f222-486b-91fb-4c0f16417e58 None admin] RBAC: Adding query filter params (name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:07.937549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2faad8-f222-486b-91fb-4c0f16417e58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:07.938949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2faad8-f222-486b-91fb-4c0f16417e58 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:07.947343 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 289/577] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jun 23 09:04:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:07.948131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2faad8-f222-486b-91fb-4c0f16417e58 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:07.994881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021791c7-8549-4910-8b0d-36285b95e6f7 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:08.033582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f3ca2-4f38-4ceb-981e-64ab0ea88b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:08.034392 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f3ca2-4f38-4ceb-981e-64ab0ea88b64 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:08.034679 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f3ca2-4f38-4ceb-981e-64ab0ea88b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:08.034986 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f3ca2-4f38-4ceb-981e-64ab0ea88b64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:08.041795 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f3ca2-4f38-4ceb-981e-64ab0ea88b64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:08.043289 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f3ca2-4f38-4ceb-981e-64ab0ea88b64 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:08.052763 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2faad8-f222-486b-91fb-4c0f16417e58 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:08.056887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-172f3ca2-4f38-4ceb-981e-64ab0ea88b64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:08.056887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:08.056887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:08.056887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:08.056887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:08.056887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:08.056887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:08.056887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:08.056887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:08.056887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:08.056887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:08.056887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:08.056887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:08.056887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:08.056887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:08.057854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:08.057854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:08.057854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:08.057854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:08.057854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:08.057854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:08.057854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:08.057854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:08.057854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:08.057854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:08.057854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:08.057854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:08.057854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:08.057854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:08.057854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:08.057854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:08.057854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:08.059505 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:08.059505 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:08.059505 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:08.059505 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:08.059505 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:08.059505 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:08.059505 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:08.059505 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:08.059505 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 289/578] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:08.060223 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f3ca2-4f38-4ceb-981e-64ab0ea88b64 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:08.086754 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:08.092949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d687619-369f-4491-8990-8a7948899148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:08.093623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d687619-369f-4491-8990-8a7948899148 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:08.093914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d687619-369f-4491-8990-8a7948899148 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:08.094197 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d687619-369f-4491-8990-8a7948899148 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:08.095081 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d687619-369f-4491-8990-8a7948899148 None admin] RBAC: Adding query filter params (role.id=edf696ae763147cab68434524d1eff9c, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=5b1331b4f0924bb8b137a22a33c9d130) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:08.095399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d687619-369f-4491-8990-8a7948899148 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:08.096894 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d687619-369f-4491-8990-8a7948899148 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:08.102493 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 290/579] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:04:07 2022] GET /identity/v3/role_assignments?user.id=5b1331b4f0924bb8b137a22a33c9d130&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=edf696ae763147cab68434524d1eff9c => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:08.103302 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d687619-369f-4491-8990-8a7948899148 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:08.229756 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:08.266662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f3ca2-4f38-4ceb-981e-64ab0ea88b64 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:08.304055 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056c09c-fee3-4520-b8c2-d59375b225f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:08.304822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056c09c-fee3-4520-b8c2-d59375b225f6 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:08.304931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d687619-369f-4491-8990-8a7948899148 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:08.305140 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056c09c-fee3-4520-b8c2-d59375b225f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:08.305387 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056c09c-fee3-4520-b8c2-d59375b225f6 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:08.306699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056c09c-fee3-4520-b8c2-d59375b225f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:08.307028 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056c09c-fee3-4520-b8c2-d59375b225f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:08.308619 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056c09c-fee3-4520-b8c2-d59375b225f6 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:08.334071 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 290/580] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jun 23 09:04:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:08.334071 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056c09c-fee3-4520-b8c2-d59375b225f6 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:08.349649 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32be31a-fa3c-4994-a41b-7519c80f2297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:08.350634 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32be31a-fa3c-4994-a41b-7519c80f2297 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:08.351949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32be31a-fa3c-4994-a41b-7519c80f2297 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:08.352251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32be31a-fa3c-4994-a41b-7519c80f2297 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:08.363857 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32be31a-fa3c-4994-a41b-7519c80f2297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:08.372771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-d32be31a-fa3c-4994-a41b-7519c80f2297 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:08.372771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:08.372771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:08.372771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:08.372771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:08.372771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:08.372771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:08.372771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:08.372771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:08.372771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:08.372771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:08.372771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:08.372771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:08.372771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:08.372771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:08.372771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:08.373868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:08.373868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:08.373868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:08.373868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:08.373868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:08.373868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:08.373868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:08.373868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:08.373868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:08.373868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:08.373868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:08.373868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:08.373868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:08.373868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:08.373868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:08.373868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:08.374779 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:08.374779 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:08.374779 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:08.374779 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:08.374779 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:08.374779 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:08.374779 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:08.374779 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:08.374779 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:08.374779 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:08.374779 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:08.374779 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:08.378303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 291/581] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:04:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:08.379006 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32be31a-fa3c-4994-a41b-7519c80f2297 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:08.448961 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:08.453935 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056c09c-fee3-4520-b8c2-d59375b225f6 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:08.489450 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:08.497303 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279fccc-9757-4ed5-be7c-2fc796897286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:08.498113 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279fccc-9757-4ed5-be7c-2fc796897286 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:08.498408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279fccc-9757-4ed5-be7c-2fc796897286 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:08.498637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32be31a-fa3c-4994-a41b-7519c80f2297 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:08.498749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279fccc-9757-4ed5-be7c-2fc796897286 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:08.509783 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279fccc-9757-4ed5-be7c-2fc796897286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:08.511297 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279fccc-9757-4ed5-be7c-2fc796897286 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:08.531298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-7279fccc-9757-4ed5-be7c-2fc796897286 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:08.531298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:08.531298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:08.531298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:08.531298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:08.531298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:08.531298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:08.531298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:08.531298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:08.531298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:08.531298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:08.531298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:08.531298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:08.531298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:08.531298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:08.532267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:08.532267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:08.532267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:08.532267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:08.532267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:08.532267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:08.532267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:08.532267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:08.532267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:08.532267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:08.532267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:08.532267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:08.532267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:08.532267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:08.532267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:08.532267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:08.532267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:08.538453 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:08.538453 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:08.538453 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:08.538453 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:08.538453 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 291/582] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:08.538453 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279fccc-9757-4ed5-be7c-2fc796897286 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:08.545715 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f6222-00d1-4015-b6ad-45cd94a57c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:08.546399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f6222-00d1-4015-b6ad-45cd94a57c8f None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:08.546748 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f6222-00d1-4015-b6ad-45cd94a57c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:08.547067 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f6222-00d1-4015-b6ad-45cd94a57c8f None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:08.547943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f6222-00d1-4015-b6ad-45cd94a57c8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:08.549921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f6222-00d1-4015-b6ad-45cd94a57c8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:08.549921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f6222-00d1-4015-b6ad-45cd94a57c8f None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:08.563329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 292/583] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:08.564158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f6222-00d1-4015-b6ad-45cd94a57c8f None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:08.658206 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279fccc-9757-4ed5-be7c-2fc796897286 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:08.694741 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f6222-00d1-4015-b6ad-45cd94a57c8f None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:08.697076 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb52a28-b787-4b80-bca0-67fb1479bbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:08.698063 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb52a28-b787-4b80-bca0-67fb1479bbcd None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:08.698358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb52a28-b787-4b80-bca0-67fb1479bbcd None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:08.698638 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb52a28-b787-4b80-bca0-67fb1479bbcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:08.712363 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb52a28-b787-4b80-bca0-67fb1479bbcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:08.713900 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb52a28-b787-4b80-bca0-67fb1479bbcd None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:08.721235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:08.734062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-3bb52a28-b787-4b80-bca0-67fb1479bbcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:08.734062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:08.734062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:08.734062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:08.734062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:08.734062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:08.734062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:08.734062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:08.734062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:08.734062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:08.734062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:08.734062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:08.734062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:08.734062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:08.734062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:08.736735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:08.736735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:08.736735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:08.736735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:08.736735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:08.736735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:08.736735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:08.736735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:08.736735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:08.736735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:08.736735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:08.736735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:08.736735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:08.736735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:08.736735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:08.736735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:08.736735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:08.737723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:08.737723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:08.737723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:08.737723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:08.737723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 292/584] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:08.737723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb52a28-b787-4b80-bca0-67fb1479bbcd None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:08.746554 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26672e48-ac1b-4a46-bb0d-9d8953e0d701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:08.747374 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26672e48-ac1b-4a46-bb0d-9d8953e0d701 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:08.747672 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26672e48-ac1b-4a46-bb0d-9d8953e0d701 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:08.747957 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26672e48-ac1b-4a46-bb0d-9d8953e0d701 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:08.748727 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26672e48-ac1b-4a46-bb0d-9d8953e0d701 None admin] RBAC: Adding query filter params (name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:08.749039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26672e48-ac1b-4a46-bb0d-9d8953e0d701 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:08.750479 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26672e48-ac1b-4a46-bb0d-9d8953e0d701 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:08.759676 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 293/585] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jun 23 09:04:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:08.760514 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26672e48-ac1b-4a46-bb0d-9d8953e0d701 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:08.784907 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:08.869170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb52a28-b787-4b80-bca0-67fb1479bbcd None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:08.885730 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26672e48-ac1b-4a46-bb0d-9d8953e0d701 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:08.910111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23565900-2473-46e7-b695-31eaf5dc1773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:08.910781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23565900-2473-46e7-b695-31eaf5dc1773 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:08.911100 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23565900-2473-46e7-b695-31eaf5dc1773 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:08.911383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23565900-2473-46e7-b695-31eaf5dc1773 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:08.912113 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23565900-2473-46e7-b695-31eaf5dc1773 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:08.912424 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23565900-2473-46e7-b695-31eaf5dc1773 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:08.913815 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23565900-2473-46e7-b695-31eaf5dc1773 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:08.923176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b1616-25ec-4505-964d-73dac0a0807f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:08.924112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b1616-25ec-4505-964d-73dac0a0807f None admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:08.924401 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b1616-25ec-4505-964d-73dac0a0807f None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:08.925098 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b1616-25ec-4505-964d-73dac0a0807f None admin] PATH_INFO: `/v3/projects/93b5040533524222b126bbe93afc56d1/users/ff025cb706d243778468ae4570374690/roles/edf696ae763147cab68434524d1eff9c` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:08.930417 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 293/586] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:08.930417 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23565900-2473-46e7-b695-31eaf5dc1773 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:08.945195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b1616-25ec-4505-964d-73dac0a0807f None admin] RBAC: Authorizing `identity:create_grant(project_id=93b5040533524222b126bbe93afc56d1, user_id=ff025cb706d243778468ae4570374690, role_id=edf696ae763147cab68434524d1eff9c)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:08.946769 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b1616-25ec-4505-964d-73dac0a0807f None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:09.019633 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:09.028705 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23565900-2473-46e7-b695-31eaf5dc1773 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:09.039269 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 294/587] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:04:08 2022] PUT /identity/v3/projects/93b5040533524222b126bbe93afc56d1/users/ff025cb706d243778468ae4570374690/roles/edf696ae763147cab68434524d1eff9c => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:04:09.043961 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b1616-25ec-4505-964d-73dac0a0807f None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:09.064166 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a75c34-3be6-48b9-8f5e-fe7ad6c1988c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:09.064905 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a75c34-3be6-48b9-8f5e-fe7ad6c1988c None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:09.065151 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a75c34-3be6-48b9-8f5e-fe7ad6c1988c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:09.065427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a75c34-3be6-48b9-8f5e-fe7ad6c1988c None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:09.066179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a75c34-3be6-48b9-8f5e-fe7ad6c1988c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:09.066490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a75c34-3be6-48b9-8f5e-fe7ad6c1988c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:09.067938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a75c34-3be6-48b9-8f5e-fe7ad6c1988c None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:09.090961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 294/588] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:09.090961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a75c34-3be6-48b9-8f5e-fe7ad6c1988c None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:09.110602 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:09.232895 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b1616-25ec-4505-964d-73dac0a0807f None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:09.235894 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a75c34-3be6-48b9-8f5e-fe7ad6c1988c None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:09.272226 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a60386-d849-4cbc-a775-0ca798dbe210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:09.273686 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a60386-d849-4cbc-a775-0ca798dbe210 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:09.274091 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c351d-cf47-4de6-aae7-12cb3a0a446e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:09.274533 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a60386-d849-4cbc-a775-0ca798dbe210 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:09.274894 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c351d-cf47-4de6-aae7-12cb3a0a446e None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:09.275224 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c351d-cf47-4de6-aae7-12cb3a0a446e None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:09.275433 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a60386-d849-4cbc-a775-0ca798dbe210 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:09.275534 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c351d-cf47-4de6-aae7-12cb3a0a446e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:09.283460 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c351d-cf47-4de6-aae7-12cb3a0a446e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:09.284938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c351d-cf47-4de6-aae7-12cb3a0a446e None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:09.287988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a60386-d849-4cbc-a775-0ca798dbe210 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:09.289710 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a60386-d849-4cbc-a775-0ca798dbe210 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:09.301005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-c86c351d-cf47-4de6-aae7-12cb3a0a446e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 09:04:09.301005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:09.301005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:09.301005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:09.301005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:09.301005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:09.301005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:09.301005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:09.301005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:09.301005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:09.301005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:09.301005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:09.301005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:09.301005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:09.301005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:09.301005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:09.302021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:09.302021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:09.302021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:09.302021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:09.302021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:09.302021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:09.302021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:09.302021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:09.302021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:09.302021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:09.302021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:09.302021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:09.302021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:09.302021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:09.302021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:09.302021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:09.302021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:09.302021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:09.303488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:09.303488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:09.303488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:09.303488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:09.303488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:09.303488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:09.303488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:09.303488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:09.303488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 09:04:09.303488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:09.303488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 295/589] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:04:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:09.307748 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c351d-cf47-4de6-aae7-12cb3a0a446e None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:09.315354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-89a60386-d849-4cbc-a775-0ca798dbe210 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:09.315354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:09.315354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:09.315354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:09.315354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:09.315354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:09.315354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:09.315354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:09.315354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:09.315354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:09.315354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:09.315354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:09.315354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:09.315354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:09.315354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:09.316398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:09.316398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:09.316398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:09.316398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:09.316398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:09.316398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:09.316398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:09.316398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:09.316398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:09.316398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:09.316398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:09.316398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:09.316398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:09.316398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:09.316398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:09.316398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:09.316398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:09.317379 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:09.317379 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:09.317379 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:09.317379 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:09.319669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 295/590] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:09.320670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a60386-d849-4cbc-a775-0ca798dbe210 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:09.323761 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:09.406752 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:09.418124 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c351d-cf47-4de6-aae7-12cb3a0a446e None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:09.426818 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a60386-d849-4cbc-a775-0ca798dbe210 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:09.455905 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6950d-6504-4788-a9d2-a86eff80752c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:09.456707 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6950d-6504-4788-a9d2-a86eff80752c None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:09.456996 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6950d-6504-4788-a9d2-a86eff80752c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:09.457329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6950d-6504-4788-a9d2-a86eff80752c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:09.464487 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988cdd3a-9413-4340-bcf8-6e63f42f8cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:09.464745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6950d-6504-4788-a9d2-a86eff80752c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:09.466139 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cdd3a-9413-4340-bcf8-6e63f42f8cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:09.466634 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cdd3a-9413-4340-bcf8-6e63f42f8cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:09.466741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6950d-6504-4788-a9d2-a86eff80752c None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:09.467419 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cdd3a-9413-4340-bcf8-6e63f42f8cc2 None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:09.468566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988cdd3a-9413-4340-bcf8-6e63f42f8cc2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:09.469080 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988cdd3a-9413-4340-bcf8-6e63f42f8cc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:09.470763 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988cdd3a-9413-4340-bcf8-6e63f42f8cc2 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:09.481195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-52a6950d-6504-4788-a9d2-a86eff80752c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:09.481195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:09.481195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:09.481195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:09.481195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:09.481195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:09.481195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:09.481195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:09.481195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:09.481195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:09.481195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:09.481195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:09.481195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:09.481195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:09.481195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:09.482157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:09.482157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:09.482157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:09.482157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:09.482157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:09.482157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:09.482157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:09.482157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:09.482157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:09.482157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:09.482157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:09.482157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:09.482157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:09.482157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:09.482157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:09.482157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:09.482157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:09.483138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:09.483138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:09.483138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:09.483138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:09.483138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:09.483138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:09.483138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:09.483138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:09.487055 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 296/591] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:09.487888 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6950d-6504-4788-a9d2-a86eff80752c None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:09.494553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 296/592] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jun 23 09:04:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:09.495747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988cdd3a-9413-4340-bcf8-6e63f42f8cc2 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:09.569612 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:09.603776 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6950d-6504-4788-a9d2-a86eff80752c None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:09.609905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988cdd3a-9413-4340-bcf8-6e63f42f8cc2 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:09.642588 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457e405-247a-4972-a15d-581ab1ab8d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:09.643293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457e405-247a-4972-a15d-581ab1ab8d9d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:09.643583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457e405-247a-4972-a15d-581ab1ab8d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:09.643865 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457e405-247a-4972-a15d-581ab1ab8d9d None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:09.644855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c457e405-247a-4972-a15d-581ab1ab8d9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:09.645052 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c457e405-247a-4972-a15d-581ab1ab8d9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:09.646407 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c457e405-247a-4972-a15d-581ab1ab8d9d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:09.658762 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5fea2-ba0f-4b11-9b29-9f38bbd08a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:09.659794 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5fea2-ba0f-4b11-9b29-9f38bbd08a62 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:09.660295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5fea2-ba0f-4b11-9b29-9f38bbd08a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:09.660763 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 297/593] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:09.660994 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5fea2-ba0f-4b11-9b29-9f38bbd08a62 None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:09.661697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457e405-247a-4972-a15d-581ab1ab8d9d None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:09.662361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5fea2-ba0f-4b11-9b29-9f38bbd08a62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:09.662986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5fea2-ba0f-4b11-9b29-9f38bbd08a62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:09.664696 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5fea2-ba0f-4b11-9b29-9f38bbd08a62 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:09.672687 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:09.685114 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 297/594] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:09.685114 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5fea2-ba0f-4b11-9b29-9f38bbd08a62 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:09.777619 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457e405-247a-4972-a15d-581ab1ab8d9d None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:09.796946 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5fea2-ba0f-4b11-9b29-9f38bbd08a62 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:09.811039 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:09.817188 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82dc08-0e48-498e-bed7-4d565eacbffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:09.824040 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82dc08-0e48-498e-bed7-4d565eacbffe None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:09.824040 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82dc08-0e48-498e-bed7-4d565eacbffe None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:09.824040 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82dc08-0e48-498e-bed7-4d565eacbffe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:09.830845 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a82dc08-0e48-498e-bed7-4d565eacbffe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:09.832544 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a82dc08-0e48-498e-bed7-4d565eacbffe None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:09.836638 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c5e73-18d6-40b9-8a90-96df2b0a4693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:09.838006 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c5e73-18d6-40b9-8a90-96df2b0a4693 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:09.838551 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c5e73-18d6-40b9-8a90-96df2b0a4693 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:09.839153 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c5e73-18d6-40b9-8a90-96df2b0a4693 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:09.847273 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c5e73-18d6-40b9-8a90-96df2b0a4693 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:09.848787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c5e73-18d6-40b9-8a90-96df2b0a4693 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:09.853491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-9a82dc08-0e48-498e-bed7-4d565eacbffe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:09.853491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:09.853491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:09.853491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:09.853491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:09.853491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:09.853491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:09.853491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:09.853491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:09.853491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:09.853491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:09.853491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:09.853491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:09.853491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:09.853491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:09.854447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:09.854447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:09.854447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:09.854447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:09.854447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:09.854447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:09.854447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:09.854447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:09.854447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:09.854447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:09.854447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:09.854447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:09.854447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:09.854447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:09.854447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:09.854447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:09.854447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:09.855476 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:09.855476 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:09.855476 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:09.855476 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:09.862469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 298/595] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:09.863496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82dc08-0e48-498e-bed7-4d565eacbffe None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:09.865069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-343c5e73-18d6-40b9-8a90-96df2b0a4693 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 09:04:09.865069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:09.865069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:09.865069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:09.865069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:09.865069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:09.865069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:09.865069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:09.865069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:09.865069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:09.865069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:09.865069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:09.865069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:09.865069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:09.865069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:09.865069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:09.866047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:09.866047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:09.866047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:09.866047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:09.866047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:09.866047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:09.866047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:09.866047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:09.866047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:09.866047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:09.866047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:09.866047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:09.866047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:09.866047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:09.866047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:09.866047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:09.866047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:09.866047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:09.867100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:09.867100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:09.867100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:09.867100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:09.867100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:09.867100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:09.867100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:09.867100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:09.867100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 09:04:09.867100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:09.867100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 298/596] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:04:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:09.867739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c5e73-18d6-40b9-8a90-96df2b0a4693 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:09.930811 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:09.976162 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c5e73-18d6-40b9-8a90-96df2b0a4693 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:09.976364 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82dc08-0e48-498e-bed7-4d565eacbffe None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:10.012347 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302ebb1-10b4-4c45-8fd0-2fc29056b10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:10.013200 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302ebb1-10b4-4c45-8fd0-2fc29056b10d None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:10.013555 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302ebb1-10b4-4c45-8fd0-2fc29056b10d None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:10.014036 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302ebb1-10b4-4c45-8fd0-2fc29056b10d None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:10.015361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5302ebb1-10b4-4c45-8fd0-2fc29056b10d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:10.015836 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd38c4b-4218-496f-9841-4a3eeff2490e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:10.015962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5302ebb1-10b4-4c45-8fd0-2fc29056b10d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:10.017040 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd38c4b-4218-496f-9841-4a3eeff2490e None admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:10.017475 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5302ebb1-10b4-4c45-8fd0-2fc29056b10d None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:10.017701 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd38c4b-4218-496f-9841-4a3eeff2490e None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:10.018164 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd38c4b-4218-496f-9841-4a3eeff2490e None admin] PATH_INFO: `/v3/projects/93b5040533524222b126bbe93afc56d1/users/36a6a3190dad4948a505bbe4c7095699/roles/edf696ae763147cab68434524d1eff9c` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:10.034826 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 299/597] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:10.034826 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302ebb1-10b4-4c45-8fd0-2fc29056b10d None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:10.039030 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd38c4b-4218-496f-9841-4a3eeff2490e None admin] RBAC: Authorizing `identity:create_grant(project_id=93b5040533524222b126bbe93afc56d1, user_id=36a6a3190dad4948a505bbe4c7095699, role_id=edf696ae763147cab68434524d1eff9c)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:10.040865 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd38c4b-4218-496f-9841-4a3eeff2490e None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:10.116755 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 299/598] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:04:09 2022] PUT /identity/v3/projects/93b5040533524222b126bbe93afc56d1/users/36a6a3190dad4948a505bbe4c7095699/roles/edf696ae763147cab68434524d1eff9c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:04:10.124063 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd38c4b-4218-496f-9841-4a3eeff2490e None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:10.161501 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302ebb1-10b4-4c45-8fd0-2fc29056b10d None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:10.194810 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:10.204639 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:10.229001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc36ff4-80dd-4909-b9db-17a3a99e61d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:10.229984 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc36ff4-80dd-4909-b9db-17a3a99e61d8 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:10.230273 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc36ff4-80dd-4909-b9db-17a3a99e61d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:10.230610 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc36ff4-80dd-4909-b9db-17a3a99e61d8 None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:10.231715 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc36ff4-80dd-4909-b9db-17a3a99e61d8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:10.232050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc36ff4-80dd-4909-b9db-17a3a99e61d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:10.234675 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc36ff4-80dd-4909-b9db-17a3a99e61d8 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:10.265297 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 300/599] 10.222.0.52 () {56 vars in 1139 bytes} [Thu Jun 23 09:04:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:10.265297 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18a0bf-4cac-4c1b-8572-107b05fca955 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:10.265297 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18a0bf-4cac-4c1b-8572-107b05fca955 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:10.265297 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18a0bf-4cac-4c1b-8572-107b05fca955 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:10.269347 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 301/600] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:10.271661 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18a0bf-4cac-4c1b-8572-107b05fca955 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:10.305160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd38c4b-4218-496f-9841-4a3eeff2490e None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:10.349122 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998f6b4-faa9-4d3a-a061-2d1f83f681f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:10.350269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998f6b4-faa9-4d3a-a061-2d1f83f681f5 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:10.350751 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998f6b4-faa9-4d3a-a061-2d1f83f681f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:10.351076 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998f6b4-faa9-4d3a-a061-2d1f83f681f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:10.358945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998f6b4-faa9-4d3a-a061-2d1f83f681f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:10.360586 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998f6b4-faa9-4d3a-a061-2d1f83f681f5 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:10.375592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-4998f6b4-faa9-4d3a-a061-2d1f83f681f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:10.375592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:10.375592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:10.375592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:10.375592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:10.375592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:10.375592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:10.375592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:10.375592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:10.375592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:10.375592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:10.375592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:10.375592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:10.375592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:10.375592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:10.376541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:10.376541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:10.376541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:10.376541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:10.376541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:10.376541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:10.376541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:10.376541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:10.376541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:10.376541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:10.376541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:10.376541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:10.376541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:10.376541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:10.376541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:10.376541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:10.376541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:10.377524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:10.377524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:10.377524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:10.377524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:10.377524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:10.377524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:10.377524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:10.377524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:10.377524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 300/601] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:10.378190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd464522-4eec-4243-97d7-01db5925776a None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:10.378693 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd464522-4eec-4243-97d7-01db5925776a None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:10.378804 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd464522-4eec-4243-97d7-01db5925776a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:10.398348 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-bd464522-4eec-4243-97d7-01db5925776a None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:10.400000 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-bd464522-4eec-4243-97d7-01db5925776a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:10.414057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18a0bf-4cac-4c1b-8572-107b05fca955 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:10.455171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd211afe-805b-4412-9c2f-4c04aef80d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:10.456308 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd211afe-805b-4412-9c2f-4c04aef80d23 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:10.456794 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd211afe-805b-4412-9c2f-4c04aef80d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:10.457323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd211afe-805b-4412-9c2f-4c04aef80d23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:10.472461 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd211afe-805b-4412-9c2f-4c04aef80d23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:10.474666 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd211afe-805b-4412-9c2f-4c04aef80d23 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:10.489899 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 301/602] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:04:10.491081 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd464522-4eec-4243-97d7-01db5925776a None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:10.506953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-dd211afe-805b-4412-9c2f-4c04aef80d23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:10.506953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:10.506953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:10.506953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:10.506953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:10.506953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:10.506953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:10.506953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:10.506953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:10.506953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:10.506953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:10.506953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:10.506953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:10.506953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:10.506953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:10.507923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:10.507923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:10.507923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:10.507923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:10.507923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:10.507923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:10.507923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:10.507923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:10.507923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:10.507923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:10.507923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:10.507923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:10.507923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:10.507923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:10.507923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:10.507923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:10.507923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:10.508887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:10.508887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:10.508887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:10.508887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:10.508887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 302/603] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:10.515945 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd211afe-805b-4412-9c2f-4c04aef80d23 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:10.522298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:10.551533 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:10.613710 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd464522-4eec-4243-97d7-01db5925776a None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:10.650938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d559411-dd02-41d7-b998-5eb988594746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:10.651541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d559411-dd02-41d7-b998-5eb988594746 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:10.651830 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d559411-dd02-41d7-b998-5eb988594746 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:10.652108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d559411-dd02-41d7-b998-5eb988594746 None admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:10.653050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d559411-dd02-41d7-b998-5eb988594746 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:10.653368 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d559411-dd02-41d7-b998-5eb988594746 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:10.653817 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd211afe-805b-4412-9c2f-4c04aef80d23 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:10.654806 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d559411-dd02-41d7-b998-5eb988594746 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:10.674396 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 302/604] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:10.674396 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d559411-dd02-41d7-b998-5eb988594746 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:10.706160 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c754100c-4346-45c8-a1ae-4de833032a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:10.706832 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754100c-4346-45c8-a1ae-4de833032a94 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:10.707188 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754100c-4346-45c8-a1ae-4de833032a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:10.707444 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754100c-4346-45c8-a1ae-4de833032a94 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:10.708168 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c754100c-4346-45c8-a1ae-4de833032a94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:10.708726 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c754100c-4346-45c8-a1ae-4de833032a94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:10.710110 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c754100c-4346-45c8-a1ae-4de833032a94 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:10.723155 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 303/605] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:10.724192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c754100c-4346-45c8-a1ae-4de833032a94 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:10.781912 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:10.807682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d559411-dd02-41d7-b998-5eb988594746 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:10.821097 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:10.834329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c754100c-4346-45c8-a1ae-4de833032a94 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:10.853922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928322e-4bf7-4d4e-8b30-39f905c29936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:10.855189 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928322e-4bf7-4d4e-8b30-39f905c29936 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:10.856711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928322e-4bf7-4d4e-8b30-39f905c29936 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:10.857182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928322e-4bf7-4d4e-8b30-39f905c29936 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:10.871039 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d928322e-4bf7-4d4e-8b30-39f905c29936 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:10.872281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe386753-8f1a-4c2e-8541-9c577eff3af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:10.873265 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe386753-8f1a-4c2e-8541-9c577eff3af9 None admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:10.873622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe386753-8f1a-4c2e-8541-9c577eff3af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:10.873968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe386753-8f1a-4c2e-8541-9c577eff3af9 None admin] PATH_INFO: `/v3/projects/93b5040533524222b126bbe93afc56d1/users/c21a758d0e2645318c89a4fe8805df0a/roles/edf696ae763147cab68434524d1eff9c` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:10.881489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-d928322e-4bf7-4d4e-8b30-39f905c29936 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:10.881489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:10.881489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:10.881489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:10.881489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:10.881489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:10.881489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:10.881489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:10.881489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:10.881489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:10.881489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:10.881489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:10.881489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:10.881489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:10.881489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:10.881489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:10.882496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:10.882496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:10.882496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:10.882496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:10.882496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:10.882496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:10.882496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:10.882496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:10.882496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:10.882496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:10.882496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:10.882496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:10.882496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:10.882496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:10.882496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:10.882496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:10.883433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:10.883433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:10.883433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:10.883433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:10.883433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:10.883433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:10.883433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:10.883433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:10.883433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:10.883433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:10.883433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:10.883433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:10.890062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 303/606] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:04:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:10.890062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928322e-4bf7-4d4e-8b30-39f905c29936 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:10.900313 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe386753-8f1a-4c2e-8541-9c577eff3af9 None admin] RBAC: Authorizing `identity:create_grant(project_id=93b5040533524222b126bbe93afc56d1, user_id=c21a758d0e2645318c89a4fe8805df0a, role_id=edf696ae763147cab68434524d1eff9c)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:10.903258 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe386753-8f1a-4c2e-8541-9c577eff3af9 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:10.999385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928322e-4bf7-4d4e-8b30-39f905c29936 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:11.042936 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 304/607] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:04:10 2022] PUT /identity/v3/projects/93b5040533524222b126bbe93afc56d1/users/c21a758d0e2645318c89a4fe8805df0a/roles/edf696ae763147cab68434524d1eff9c => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:04:11.042936 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe386753-8f1a-4c2e-8541-9c577eff3af9 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:11.049066 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354132d5-b1a6-4222-a232-6e0d1cd5a82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:11.049977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354132d5-b1a6-4222-a232-6e0d1cd5a82a None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:11.057590 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354132d5-b1a6-4222-a232-6e0d1cd5a82a None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:11.057590 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354132d5-b1a6-4222-a232-6e0d1cd5a82a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:11.068043 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354132d5-b1a6-4222-a232-6e0d1cd5a82a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:11.069657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354132d5-b1a6-4222-a232-6e0d1cd5a82a None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:11.099699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-354132d5-b1a6-4222-a232-6e0d1cd5a82a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:11.099699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:11.099699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:11.099699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:11.099699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:11.099699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:11.099699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:11.099699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:11.099699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:11.099699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:11.099699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:11.099699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:11.099699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:11.099699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:11.099699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:11.100745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:11.100745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:11.100745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:11.100745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:11.100745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:11.100745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:11.100745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:11.100745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:11.100745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:11.100745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:11.100745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:11.100745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:11.100745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:11.100745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:11.100745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:11.100745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:11.100745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:11.101702 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:11.101702 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:11.101702 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:11.101702 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:11.101702 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:11.101702 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:11.101702 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:11.101702 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:11.101702 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 304/608] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:11.107856 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37307837-8ee5-48e8-9497-0067872356b7 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:11.108161 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37307837-8ee5-48e8-9497-0067872356b7 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:11.108445 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37307837-8ee5-48e8-9497-0067872356b7 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:11.110030 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 305/609] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:11.111008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37307837-8ee5-48e8-9497-0067872356b7 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:11.117489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:11.201739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe386753-8f1a-4c2e-8541-9c577eff3af9 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:11.241512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e1f3b-5188-4134-b2a6-657348d13e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:11.242192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e1f3b-5188-4134-b2a6-657348d13e0f None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:11.242479 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e1f3b-5188-4134-b2a6-657348d13e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:11.242759 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e1f3b-5188-4134-b2a6-657348d13e0f None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:11.243546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e1f3b-5188-4134-b2a6-657348d13e0f None admin] RBAC: Adding query filter params (name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:11.243855 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e1f3b-5188-4134-b2a6-657348d13e0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:11.245424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e1f3b-5188-4134-b2a6-657348d13e0f None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:11.245788 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37307837-8ee5-48e8-9497-0067872356b7 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:11.250899 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:11.255438 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 305/610] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jun 23 09:04:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:11.256706 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71015baa-7ce9-4abb-b901-875d4499866c None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:11.257128 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71015baa-7ce9-4abb-b901-875d4499866c None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:11.257408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71015baa-7ce9-4abb-b901-875d4499866c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:11.275848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-71015baa-7ce9-4abb-b901-875d4499866c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:11.278081 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-71015baa-7ce9-4abb-b901-875d4499866c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:11.289479 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd8384-2268-4594-be61-c7fbb03da2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:11.290158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd8384-2268-4594-be61-c7fbb03da2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:11.290450 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd8384-2268-4594-be61-c7fbb03da2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:11.290749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd8384-2268-4594-be61-c7fbb03da2a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:11.291557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd8384-2268-4594-be61-c7fbb03da2a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:11.291870 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd8384-2268-4594-be61-c7fbb03da2a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:11.294139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd8384-2268-4594-be61-c7fbb03da2a7 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:11.306312 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 306/611] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:11.307169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd8384-2268-4594-be61-c7fbb03da2a7 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:11.355011 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 306/612] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:04:11.356799 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71015baa-7ce9-4abb-b901-875d4499866c None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:11.375465 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:11.415493 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd8384-2268-4594-be61-c7fbb03da2a7 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:11.451027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8568b-1ad9-43b1-a2a8-9b1fdfc9e506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:11.451847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8568b-1ad9-43b1-a2a8-9b1fdfc9e506 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:11.452142 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8568b-1ad9-43b1-a2a8-9b1fdfc9e506 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:11.452428 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8568b-1ad9-43b1-a2a8-9b1fdfc9e506 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:11.463722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8568b-1ad9-43b1-a2a8-9b1fdfc9e506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:11.465312 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8568b-1ad9-43b1-a2a8-9b1fdfc9e506 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:11.478055 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71015baa-7ce9-4abb-b901-875d4499866c None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:11.489506 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-b2e8568b-1ad9-43b1-a2a8-9b1fdfc9e506 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:11.489506 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:11.489506 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:11.489506 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:11.489506 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:11.489506 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:11.489506 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:11.489506 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:11.489506 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:11.489506 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:11.489506 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:11.489506 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:11.489506 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:11.489506 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:11.489506 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:11.490470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:11.490470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:11.490470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:11.490470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:11.490470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:11.490470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:11.490470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:11.490470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:11.490470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:11.490470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:11.490470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:11.490470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:11.490470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:11.490470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:11.490470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:11.490470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:11.490470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:11.491744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:11.491744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:11.491744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:11.491744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:11.491744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 307/613] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:11.495101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8568b-1ad9-43b1-a2a8-9b1fdfc9e506 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:11.523294 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:11.532644 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825e491-7bdd-4c6d-9353-4cba3ddd1d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:11.534503 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825e491-7bdd-4c6d-9353-4cba3ddd1d43 None admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:11.534795 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825e491-7bdd-4c6d-9353-4cba3ddd1d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:11.535108 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825e491-7bdd-4c6d-9353-4cba3ddd1d43 None admin] PATH_INFO: `/v3/projects/93b5040533524222b126bbe93afc56d1/users/daed809f511a43ed8d3f88eba10466b2/roles/edf696ae763147cab68434524d1eff9c` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:11.552692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d825e491-7bdd-4c6d-9353-4cba3ddd1d43 None admin] RBAC: Authorizing `identity:create_grant(project_id=93b5040533524222b126bbe93afc56d1, user_id=daed809f511a43ed8d3f88eba10466b2, role_id=edf696ae763147cab68434524d1eff9c)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:11.554364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d825e491-7bdd-4c6d-9353-4cba3ddd1d43 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:11.606857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8568b-1ad9-43b1-a2a8-9b1fdfc9e506 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:11.624079 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:11.641639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d6d08-4992-40c2-8485-9301164eef1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:11.642441 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d6d08-4992-40c2-8485-9301164eef1b None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:11.642729 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d6d08-4992-40c2-8485-9301164eef1b None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:11.643040 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d6d08-4992-40c2-8485-9301164eef1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:11.655635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d6d08-4992-40c2-8485-9301164eef1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:11.665097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-f62d6d08-4992-40c2-8485-9301164eef1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:11.665097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:11.665097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:11.665097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:11.665097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:11.665097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:11.665097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:11.665097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:11.665097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:11.665097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:11.665097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:11.665097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:11.665097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:11.665097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:11.665097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:11.665097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:11.671517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:11.671517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:11.671517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:11.671517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:11.671517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:11.671517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:11.671517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:11.671517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:11.671517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:11.671517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:11.671517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:11.671517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:11.671517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:11.671517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:11.671517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:11.671517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:11.672661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:11.672661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:11.672661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:11.672661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:11.672661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:11.672661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:11.672661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:11.672661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:11.672661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:11.672661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:11.672661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:11.672661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:11.672661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 308/615] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:04:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:11.672661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d6d08-4992-40c2-8485-9301164eef1b None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:11.673497 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 307/615] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:04:11 2022] PUT /identity/v3/projects/93b5040533524222b126bbe93afc56d1/users/daed809f511a43ed8d3f88eba10466b2/roles/edf696ae763147cab68434524d1eff9c => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:04:11.673497 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825e491-7bdd-4c6d-9353-4cba3ddd1d43 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:11.804770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d6d08-4992-40c2-8485-9301164eef1b None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:11.848771 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763175fd-42e8-49c2-8473-fa75546416e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:11.849757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763175fd-42e8-49c2-8473-fa75546416e8 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:11.850250 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763175fd-42e8-49c2-8473-fa75546416e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:11.850743 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763175fd-42e8-49c2-8473-fa75546416e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:11.851921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763175fd-42e8-49c2-8473-fa75546416e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:11.852387 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763175fd-42e8-49c2-8473-fa75546416e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:11.854247 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763175fd-42e8-49c2-8473-fa75546416e8 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:11.870377 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 309/616] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:11.880212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763175fd-42e8-49c2-8473-fa75546416e8 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:11.920858 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825e491-7bdd-4c6d-9353-4cba3ddd1d43 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:11.949083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:11.978090 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f8f06-7cd7-4fd8-b05f-14a941e0a348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:11.979656 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f8f06-7cd7-4fd8-b05f-14a941e0a348 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:11.980124 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f8f06-7cd7-4fd8-b05f-14a941e0a348 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:11.980788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f8f06-7cd7-4fd8-b05f-14a941e0a348 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:11.982041 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429f8f06-7cd7-4fd8-b05f-14a941e0a348 None admin] RBAC: Adding query filter params (name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:11.982776 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429f8f06-7cd7-4fd8-b05f-14a941e0a348 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:11.984812 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429f8f06-7cd7-4fd8-b05f-14a941e0a348 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:11.998120 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 308/617] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jun 23 09:04:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:12.004346 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f8f06-7cd7-4fd8-b05f-14a941e0a348 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:12.037855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763175fd-42e8-49c2-8473-fa75546416e8 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:12.088214 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:12.111635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776fccd-80d7-43c7-9289-73e5d696d63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:12.112740 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776fccd-80d7-43c7-9289-73e5d696d63c None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:12.114036 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776fccd-80d7-43c7-9289-73e5d696d63c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:12.114475 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776fccd-80d7-43c7-9289-73e5d696d63c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:12.129168 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7776fccd-80d7-43c7-9289-73e5d696d63c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:12.132864 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7776fccd-80d7-43c7-9289-73e5d696d63c None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:12.159449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-7776fccd-80d7-43c7-9289-73e5d696d63c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 09:04:12.159449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:12.159449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:12.159449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:12.159449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:12.159449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:12.159449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:12.159449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:12.159449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:12.159449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:12.159449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:12.159449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:12.159449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:12.159449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:12.159449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:12.159449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:12.160653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:12.160653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:12.160653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:12.160653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:12.160653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:12.160653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:12.160653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:12.160653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:12.160653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:12.160653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:12.160653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:12.160653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:12.160653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:12.160653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:12.160653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:12.160653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:12.160653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:12.160653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:12.162061 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:12.162061 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:12.162061 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:12.162061 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:12.162061 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:12.162061 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:12.162061 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:12.162061 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:12.162061 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 09:04:12.162061 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:12.162697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 310/618] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:04:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:12.172155 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776fccd-80d7-43c7-9289-73e5d696d63c None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:12.213582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f8f06-7cd7-4fd8-b05f-14a941e0a348 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:12.286869 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38eec2-6622-433e-b76b-15b29e5075b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:12.287709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38eec2-6622-433e-b76b-15b29e5075b0 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:12.288001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38eec2-6622-433e-b76b-15b29e5075b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:12.288288 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38eec2-6622-433e-b76b-15b29e5075b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:12.313033 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38eec2-6622-433e-b76b-15b29e5075b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:12.314787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38eec2-6622-433e-b76b-15b29e5075b0 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:12.340794 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:12.347385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-df38eec2-6622-433e-b76b-15b29e5075b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:12.347385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:12.347385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:12.347385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:12.347385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:12.347385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:12.347385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:12.347385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:12.347385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:12.347385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:12.347385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:12.347385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:12.347385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:12.347385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:12.347385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:12.348354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:12.348354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:12.348354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:12.348354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:12.348354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:12.348354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:12.348354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:12.348354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:12.348354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:12.348354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:12.348354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:12.348354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:12.348354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:12.348354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:12.348354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:12.348354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:12.348354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:12.349705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:12.349705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:12.349705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:12.349705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:12.349705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 309/619] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:12.350348 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776fccd-80d7-43c7-9289-73e5d696d63c None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:12.359915 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38eec2-6622-433e-b76b-15b29e5075b0 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:12.390947 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e2153-6f6a-475a-ba17-4f477a90a721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:12.390947 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e2153-6f6a-475a-ba17-4f477a90a721 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:12.390947 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e2153-6f6a-475a-ba17-4f477a90a721 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:12.390947 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e2153-6f6a-475a-ba17-4f477a90a721 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:12.391393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e2153-6f6a-475a-ba17-4f477a90a721 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:12.391393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e2153-6f6a-475a-ba17-4f477a90a721 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:12.391881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e2153-6f6a-475a-ba17-4f477a90a721 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:12.415433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 311/620] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jun 23 09:04:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:12.417218 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddaba1f-a67e-44fc-9f42-cbe34b42e570 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:12.417521 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddaba1f-a67e-44fc-9f42-cbe34b42e570 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:12.417806 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddaba1f-a67e-44fc-9f42-cbe34b42e570 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:12.419591 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 312/621] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:12.422863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddaba1f-a67e-44fc-9f42-cbe34b42e570 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:12.494369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:12.502794 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38eec2-6622-433e-b76b-15b29e5075b0 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:12.538550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b804e32-f6fa-40c4-8c5f-2cc9733dcabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:12.539635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b804e32-f6fa-40c4-8c5f-2cc9733dcabd None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:12.540146 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b804e32-f6fa-40c4-8c5f-2cc9733dcabd None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:12.540644 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b804e32-f6fa-40c4-8c5f-2cc9733dcabd None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:12.541622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b804e32-f6fa-40c4-8c5f-2cc9733dcabd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:12.542135 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b804e32-f6fa-40c4-8c5f-2cc9733dcabd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:12.544358 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b804e32-f6fa-40c4-8c5f-2cc9733dcabd None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:12.556903 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddaba1f-a67e-44fc-9f42-cbe34b42e570 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:12.560436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 310/622] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:12.561974 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529bb55-55c3-4bbd-a9bc-d3dab6db1aba None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:12.562478 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529bb55-55c3-4bbd-a9bc-d3dab6db1aba None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:12.562994 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529bb55-55c3-4bbd-a9bc-d3dab6db1aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:12.580659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-d529bb55-55c3-4bbd-a9bc-d3dab6db1aba None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:12.582157 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-d529bb55-55c3-4bbd-a9bc-d3dab6db1aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:12.603138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74943d93-6c83-43db-ac19-13022cd140c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:12.604292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74943d93-6c83-43db-ac19-13022cd140c4 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:12.605012 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74943d93-6c83-43db-ac19-13022cd140c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:12.605311 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74943d93-6c83-43db-ac19-13022cd140c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:12.620989 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74943d93-6c83-43db-ac19-13022cd140c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:12.623177 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74943d93-6c83-43db-ac19-13022cd140c4 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:12.645353 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-74943d93-6c83-43db-ac19-13022cd140c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:12.645353 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:12.645353 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:12.645353 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:12.645353 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:12.645353 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:12.645353 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:12.645353 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:12.645353 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:12.645353 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:12.645353 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:12.645353 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:12.645353 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:12.645353 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:12.645353 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:12.646358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:12.646358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:12.646358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:12.646358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:12.646358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:12.646358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:12.646358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:12.646358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:12.646358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:12.646358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:12.646358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:12.646358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:12.646358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:12.646358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:12.646358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:12.646358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:12.646358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:12.647365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:12.647365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:12.647365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:12.647365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:12.654366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 313/623] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:12.655533 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74943d93-6c83-43db-ac19-13022cd140c4 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:12.664959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:12.670931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 311/624] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:04:12.670931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529bb55-55c3-4bbd-a9bc-d3dab6db1aba None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:12.779670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:12.784948 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74943d93-6c83-43db-ac19-13022cd140c4 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:12.800361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529bb55-55c3-4bbd-a9bc-d3dab6db1aba None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:12.823656 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc803b9-8ea0-4e8f-8b8d-a3040995d567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:12.825392 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc803b9-8ea0-4e8f-8b8d-a3040995d567 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:12.825884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc803b9-8ea0-4e8f-8b8d-a3040995d567 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:12.826363 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc803b9-8ea0-4e8f-8b8d-a3040995d567 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:12.834989 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc803b9-8ea0-4e8f-8b8d-a3040995d567 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:12.836802 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc803b9-8ea0-4e8f-8b8d-a3040995d567 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:12.844589 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484d900-22bc-4011-b3e6-225756d05342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:12.845266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484d900-22bc-4011-b3e6-225756d05342 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:12.845556 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484d900-22bc-4011-b3e6-225756d05342 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:12.845839 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484d900-22bc-4011-b3e6-225756d05342 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:12.846571 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484d900-22bc-4011-b3e6-225756d05342 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:12.846881 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484d900-22bc-4011-b3e6-225756d05342 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:12.848311 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484d900-22bc-4011-b3e6-225756d05342 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:12.853926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-0cc803b9-8ea0-4e8f-8b8d-a3040995d567 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 09:04:12.853926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:12.853926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:12.853926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:12.853926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:12.853926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:12.853926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:12.853926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:12.853926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:12.853926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:12.853926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:12.853926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:12.853926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:12.853926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:12.853926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:12.853926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:12.854901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:12.854901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:12.854901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:12.854901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:12.854901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:12.854901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:12.854901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:12.854901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:12.854901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:12.854901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:12.854901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:12.854901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:12.854901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:12.854901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:12.854901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:12.854901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:12.854901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:12.854901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:12.856484 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:12.856484 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:12.856484 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:12.856484 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:12.856484 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:12.856484 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:12.856484 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:12.856484 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:12.856484 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 09:04:12.856484 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:12.859320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 314/625] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jun 23 09:04:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:12.860393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc803b9-8ea0-4e8f-8b8d-a3040995d567 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:12.865135 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 312/626] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:12.865135 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484d900-22bc-4011-b3e6-225756d05342 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:12.936925 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:12.976433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc803b9-8ea0-4e8f-8b8d-a3040995d567 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:13.003578 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484d900-22bc-4011-b3e6-225756d05342 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:13.017394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11465a6-0de3-4b0a-9325-864e0e966b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:13.018372 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11465a6-0de3-4b0a-9325-864e0e966b74 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:13.018665 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11465a6-0de3-4b0a-9325-864e0e966b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:13.018967 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11465a6-0de3-4b0a-9325-864e0e966b74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:13.034084 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11465a6-0de3-4b0a-9325-864e0e966b74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:13.041117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0ffc4-6fec-4fec-a39a-81a1f99d00fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:13.042154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0ffc4-6fec-4fec-a39a-81a1f99d00fc None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:13.042677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0ffc4-6fec-4fec-a39a-81a1f99d00fc None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:13.043271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0ffc4-6fec-4fec-a39a-81a1f99d00fc None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:13.043959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-b11465a6-0de3-4b0a-9325-864e0e966b74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:13.043959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:13.043959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:13.043959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:13.043959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:13.043959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:13.043959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:13.043959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:13.043959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:13.043959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:13.043959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:13.043959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:13.043959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:13.043959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:13.043959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:13.043959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:13.044943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:13.044943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:13.044943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:13.044943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:13.044943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:13.044943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:13.044943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:13.044943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:13.044943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:13.044943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:13.044943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:13.044943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:13.044943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:13.044943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:13.044943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:13.044943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:13.045860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:13.045860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:13.045860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:13.045860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:13.045860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:13.045860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:13.045860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:13.045860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:13.045860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:13.045860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:13.045860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:13.045860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:13.045860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 315/627] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:04:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:13.046625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11465a6-0de3-4b0a-9325-864e0e966b74 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:13.052209 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0ffc4-6fec-4fec-a39a-81a1f99d00fc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:13.052842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0ffc4-6fec-4fec-a39a-81a1f99d00fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:13.054591 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0ffc4-6fec-4fec-a39a-81a1f99d00fc None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:13.075124 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 313/628] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jun 23 09:04:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:13.080968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0ffc4-6fec-4fec-a39a-81a1f99d00fc None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:13.080968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:13.177926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11465a6-0de3-4b0a-9325-864e0e966b74 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:13.197345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0ffc4-6fec-4fec-a39a-81a1f99d00fc None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:13.206465 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:13.215451 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31988bba-3d5c-44c3-b407-66c47e501441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:13.216260 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31988bba-3d5c-44c3-b407-66c47e501441 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:13.216553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31988bba-3d5c-44c3-b407-66c47e501441 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:13.216837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31988bba-3d5c-44c3-b407-66c47e501441 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:13.223688 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31988bba-3d5c-44c3-b407-66c47e501441 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:13.235559 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31988bba-3d5c-44c3-b407-66c47e501441 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:13.237444 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3a4d4-7697-4496-a4f7-8611cd471d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:13.238321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3a4d4-7697-4496-a4f7-8611cd471d31 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:13.238796 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3a4d4-7697-4496-a4f7-8611cd471d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:13.239290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3a4d4-7697-4496-a4f7-8611cd471d31 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:13.240017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-31988bba-3d5c-44c3-b407-66c47e501441 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:13.240017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:13.240017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:13.240017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:13.240017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:13.240017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:13.240017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:13.240017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:13.240017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:13.240017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:13.240017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:13.240017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:13.240017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:13.240017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:13.240017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:13.240945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:13.240945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:13.240945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:13.240945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:13.240945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:13.240945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:13.240945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:13.240945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:13.240945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:13.240945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:13.240945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:13.240945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:13.240945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:13.240945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:13.240945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:13.240945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:13.240945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:13.241900 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:13.241900 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:13.241900 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:13.241900 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:13.241900 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:13.241900 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:13.241900 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:13.241900 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:13.241900 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 316/629] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:13.242428 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31988bba-3d5c-44c3-b407-66c47e501441 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:13.243112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3a4d4-7697-4496-a4f7-8611cd471d31 None admin] RBAC: Adding query filter params (name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:13.243438 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3a4d4-7697-4496-a4f7-8611cd471d31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:13.244809 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3a4d4-7697-4496-a4f7-8611cd471d31 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:13.255454 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 314/630] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jun 23 09:04:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:13.256721 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ba4d1-83b2-4894-95d2-b4bc255d8a63 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:13.257016 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ba4d1-83b2-4894-95d2-b4bc255d8a63 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:13.257305 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ba4d1-83b2-4894-95d2-b4bc255d8a63 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:13.259493 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 315/631] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:13.260499 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ba4d1-83b2-4894-95d2-b4bc255d8a63 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:13.338723 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:13.362790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31988bba-3d5c-44c3-b407-66c47e501441 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:13.380544 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ba4d1-83b2-4894-95d2-b4bc255d8a63 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:13.399876 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bc21a-3035-4662-9897-1e6348ceb5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:13.400534 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bc21a-3035-4662-9897-1e6348ceb5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:13.400825 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bc21a-3035-4662-9897-1e6348ceb5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:13.401107 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bc21a-3035-4662-9897-1e6348ceb5d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:13.418672 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20760fd-6120-4175-b0cd-90967781b33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:13.419897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20760fd-6120-4175-b0cd-90967781b33e None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:13.420671 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20760fd-6120-4175-b0cd-90967781b33e None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:13.421140 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20760fd-6120-4175-b0cd-90967781b33e None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:13.422211 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20760fd-6120-4175-b0cd-90967781b33e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:13.422707 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20760fd-6120-4175-b0cd-90967781b33e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:13.425048 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20760fd-6120-4175-b0cd-90967781b33e None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:13.441994 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 316/632] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:13.443942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6847de9-d2fa-4081-b574-b2dcc6dea8db None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:13.444719 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6847de9-d2fa-4081-b574-b2dcc6dea8db None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:13.445187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6847de9-d2fa-4081-b574-b2dcc6dea8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:13.456780 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:13.467210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-d6847de9-d2fa-4081-b574-b2dcc6dea8db None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:13.469153 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-d6847de9-d2fa-4081-b574-b2dcc6dea8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:13.470530 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5bc21a-3035-4662-9897-1e6348ceb5d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:13.472045 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5bc21a-3035-4662-9897-1e6348ceb5d0 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:13.542513 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 317/633] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:04:13.543769 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6847de9-d2fa-4081-b574-b2dcc6dea8db None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:13.594599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 317/634] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:04:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:04:13.595387 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bc21a-3035-4662-9897-1e6348ceb5d0 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:13.612622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:13.662685 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6847de9-d2fa-4081-b574-b2dcc6dea8db None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:13.700334 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250f92d-3835-4a17-89a0-37e8a5596d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:13.701140 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250f92d-3835-4a17-89a0-37e8a5596d3e None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:13.701426 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250f92d-3835-4a17-89a0-37e8a5596d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:13.701705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250f92d-3835-4a17-89a0-37e8a5596d3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:13.705453 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:13.713159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250f92d-3835-4a17-89a0-37e8a5596d3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:13.714854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250f92d-3835-4a17-89a0-37e8a5596d3e None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:13.724344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bc21a-3035-4662-9897-1e6348ceb5d0 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:13.733908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-8250f92d-3835-4a17-89a0-37e8a5596d3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:13.733908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:13.733908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:13.733908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:13.733908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:13.733908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:13.733908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:13.733908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:13.733908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:13.733908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:13.733908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:13.733908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:13.733908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:13.733908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:13.733908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:13.734845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:13.734845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:13.734845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:13.734845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:13.734845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:13.734845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:13.734845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:13.734845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:13.734845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:13.734845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:13.734845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:13.734845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:13.734845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:13.734845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:13.734845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:13.734845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:13.734845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:13.735923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:13.735923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:13.735923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:13.735923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:13.735923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 318/635] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:13.735923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250f92d-3835-4a17-89a0-37e8a5596d3e None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:13.763721 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b457338-80e2-4bfc-8e58-0b93780f1390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:13.764622 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b457338-80e2-4bfc-8e58-0b93780f1390 None admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:13.764910 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b457338-80e2-4bfc-8e58-0b93780f1390 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:13.765189 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b457338-80e2-4bfc-8e58-0b93780f1390 None admin] PATH_INFO: `/v3/projects/93b5040533524222b126bbe93afc56d1/users/5b1331b4f0924bb8b137a22a33c9d130/roles/edf696ae763147cab68434524d1eff9c` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:13.780179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b457338-80e2-4bfc-8e58-0b93780f1390 None admin] RBAC: Authorizing `identity:create_grant(project_id=93b5040533524222b126bbe93afc56d1, user_id=5b1331b4f0924bb8b137a22a33c9d130, role_id=edf696ae763147cab68434524d1eff9c)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:13.781640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b457338-80e2-4bfc-8e58-0b93780f1390 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:13.847321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250f92d-3835-4a17-89a0-37e8a5596d3e None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:13.854797 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:13.881689 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430cdd51-e70c-4c02-af25-341e5d544724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:13.882544 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430cdd51-e70c-4c02-af25-341e5d544724 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:13.882826 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430cdd51-e70c-4c02-af25-341e5d544724 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:13.883145 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430cdd51-e70c-4c02-af25-341e5d544724 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:13.891203 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 318/636] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:04:13 2022] PUT /identity/v3/projects/93b5040533524222b126bbe93afc56d1/users/5b1331b4f0924bb8b137a22a33c9d130/roles/edf696ae763147cab68434524d1eff9c => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:04:13.892116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b457338-80e2-4bfc-8e58-0b93780f1390 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:13.899664 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430cdd51-e70c-4c02-af25-341e5d544724 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:13.911167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-430cdd51-e70c-4c02-af25-341e5d544724 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:13.911167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:13.911167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:13.911167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:13.911167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:13.911167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:13.911167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:13.911167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:13.911167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:13.911167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:13.911167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:13.911167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:13.911167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:13.911167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:13.911167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:13.911167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:13.912264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:13.912264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:13.912264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:13.912264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:13.912264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:13.912264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:13.912264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:13.912264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:13.912264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:13.912264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:13.912264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:13.912264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:13.912264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:13.912264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:13.912264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:13.912264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:13.913158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:13.913158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:13.913158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:13.913158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:13.913158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:13.913158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:13.913158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:13.913158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:13.913158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:13.913158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:13.913158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:13.913158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:13.923814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 319/637] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:04:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:13.925438 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430cdd51-e70c-4c02-af25-341e5d544724 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:14.035997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b457338-80e2-4bfc-8e58-0b93780f1390 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:14.060522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430cdd51-e70c-4c02-af25-341e5d544724 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:14.077074 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0c005-0fee-4da7-96bd-0f627a5ff0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:14.078003 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0c005-0fee-4da7-96bd-0f627a5ff0de None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:14.078275 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0c005-0fee-4da7-96bd-0f627a5ff0de None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:14.078545 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0c005-0fee-4da7-96bd-0f627a5ff0de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:14.081444 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:14.090520 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0c005-0fee-4da7-96bd-0f627a5ff0de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:14.092116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0c005-0fee-4da7-96bd-0f627a5ff0de None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:14.102417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8822be1-606a-46ce-b5f6-95f6d0d13fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:14.102417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8822be1-606a-46ce-b5f6-95f6d0d13fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:14.102417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8822be1-606a-46ce-b5f6-95f6d0d13fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:14.102417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8822be1-606a-46ce-b5f6-95f6d0d13fc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:14.102784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8822be1-606a-46ce-b5f6-95f6d0d13fc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:14.102784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8822be1-606a-46ce-b5f6-95f6d0d13fc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:14.102784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8822be1-606a-46ce-b5f6-95f6d0d13fc0 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:14.108915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-f0d0c005-0fee-4da7-96bd-0f627a5ff0de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:14.108915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:14.108915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:14.108915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:14.108915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:14.108915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:14.108915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:14.108915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:14.108915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:14.108915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:14.108915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:14.108915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:14.108915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:14.108915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:14.108915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:14.109957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:14.109957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:14.109957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:14.109957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:14.109957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:14.109957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:14.109957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:14.109957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:14.109957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:14.109957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:14.109957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:14.109957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:14.109957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:14.109957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:14.109957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:14.109957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:14.109957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:14.110901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:14.110901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:14.110901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:14.110901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:14.110901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 319/638] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:14.110901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0c005-0fee-4da7-96bd-0f627a5ff0de None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:14.117511 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 320/639] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:14.119129 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b0ad3-91af-4593-a64b-9fea91fbf1fa None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:14.119634 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b0ad3-91af-4593-a64b-9fea91fbf1fa None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:14.120140 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b0ad3-91af-4593-a64b-9fea91fbf1fa None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:14.122382 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 321/640] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:14.123416 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b0ad3-91af-4593-a64b-9fea91fbf1fa None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:14.155804 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:14.231349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b0ad3-91af-4593-a64b-9fea91fbf1fa None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:14.238125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0c005-0fee-4da7-96bd-0f627a5ff0de None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:14.266095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de1dae-f0ec-4483-85e4-e89c26dc3a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:14.266764 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de1dae-f0ec-4483-85e4-e89c26dc3a60 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:14.267090 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de1dae-f0ec-4483-85e4-e89c26dc3a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:14.267365 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de1dae-f0ec-4483-85e4-e89c26dc3a60 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:14.268106 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de1dae-f0ec-4483-85e4-e89c26dc3a60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:14.268414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de1dae-f0ec-4483-85e4-e89c26dc3a60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:14.269800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de1dae-f0ec-4483-85e4-e89c26dc3a60 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:14.275879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174a3ac-eec0-4bc6-afc7-e5f4b090791a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:14.276567 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174a3ac-eec0-4bc6-afc7-e5f4b090791a None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:14.276859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174a3ac-eec0-4bc6-afc7-e5f4b090791a None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:14.277144 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174a3ac-eec0-4bc6-afc7-e5f4b090791a None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:14.278330 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174a3ac-eec0-4bc6-afc7-e5f4b090791a None admin] RBAC: Adding query filter params (name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:14.278789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174a3ac-eec0-4bc6-afc7-e5f4b090791a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:14.280907 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174a3ac-eec0-4bc6-afc7-e5f4b090791a None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:14.287002 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 322/641] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:14.287002 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de1dae-f0ec-4483-85e4-e89c26dc3a60 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:14.294890 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 320/642] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jun 23 09:04:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:14.296367 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9d173-8371-41ab-80de-5eff6b5fad23 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:14.296773 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9d173-8371-41ab-80de-5eff6b5fad23 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:14.297125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9d173-8371-41ab-80de-5eff6b5fad23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:14.314495 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-a1e9d173-8371-41ab-80de-5eff6b5fad23 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:14.316118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a1e9d173-8371-41ab-80de-5eff6b5fad23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:14.366889 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:14.377602 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:14.388367 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 321/643] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:04:14.389214 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9d173-8371-41ab-80de-5eff6b5fad23 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:14.406600 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de1dae-f0ec-4483-85e4-e89c26dc3a60 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:14.444515 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10920941-6ab5-4918-9bf2-37e316004be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:14.445325 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10920941-6ab5-4918-9bf2-37e316004be5 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:14.445617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10920941-6ab5-4918-9bf2-37e316004be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:14.445897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10920941-6ab5-4918-9bf2-37e316004be5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:14.453653 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10920941-6ab5-4918-9bf2-37e316004be5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:14.455434 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10920941-6ab5-4918-9bf2-37e316004be5 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:14.470404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-10920941-6ab5-4918-9bf2-37e316004be5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 09:04:14.470404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:14.470404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:14.470404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:14.470404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:14.470404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:14.470404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:14.470404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:14.470404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:14.470404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:14.470404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:14.470404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:14.470404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:14.470404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:14.470404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:14.470404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:14.471718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:14.471718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:14.471718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:14.471718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:14.471718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:14.471718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:14.471718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:14.471718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:14.471718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:14.471718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:14.471718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:14.471718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:14.471718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:14.471718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:14.471718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:14.471718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:14.471718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:14.471718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:14.472957 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:14.472957 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:14.472957 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:14.472957 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:14.472957 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:14.472957 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:14.472957 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:14.472957 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:14.472957 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 09:04:14.472957 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:14.476224 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 323/644] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:04:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:14.476979 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10920941-6ab5-4918-9bf2-37e316004be5 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:14.511651 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9d173-8371-41ab-80de-5eff6b5fad23 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:14.550951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c79569-fbf4-4363-8a8d-c3b513fbcc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:14.551868 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c79569-fbf4-4363-8a8d-c3b513fbcc4c None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:14.551984 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c79569-fbf4-4363-8a8d-c3b513fbcc4c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:14.552219 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c79569-fbf4-4363-8a8d-c3b513fbcc4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:14.559606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c79569-fbf4-4363-8a8d-c3b513fbcc4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:14.561076 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c79569-fbf4-4363-8a8d-c3b513fbcc4c None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:14.576158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-66c79569-fbf4-4363-8a8d-c3b513fbcc4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:14.576158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:14.576158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:14.576158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:14.576158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:14.576158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:14.576158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:14.576158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:14.576158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:14.576158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:14.576158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:14.576158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:14.576158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:14.576158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:14.576158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:14.577344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:14.577344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:14.577344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:14.577344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:14.577344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:14.577344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:14.577344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:14.577344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:14.577344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:14.577344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:14.577344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:14.577344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:14.577344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:14.577344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:14.577344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:14.577344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:14.577344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:14.578463 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:14.578463 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:14.578463 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:14.578463 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:14.578463 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:14.578463 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:14.578463 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:14.578463 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:14.578463 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 322/645] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:14.578463 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c79569-fbf4-4363-8a8d-c3b513fbcc4c None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:14.588200 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10920941-6ab5-4918-9bf2-37e316004be5 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:14.623924 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06816cd5-47b6-4445-941e-e5e88ef39241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:14.624654 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06816cd5-47b6-4445-941e-e5e88ef39241 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:14.624944 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06816cd5-47b6-4445-941e-e5e88ef39241 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:14.625084 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:14.625212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06816cd5-47b6-4445-941e-e5e88ef39241 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:14.637370 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06816cd5-47b6-4445-941e-e5e88ef39241 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:14.638821 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06816cd5-47b6-4445-941e-e5e88ef39241 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:14.645769 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:14.657154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-06816cd5-47b6-4445-941e-e5e88ef39241 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:14.657154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:14.657154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:14.657154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:14.657154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:14.657154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:14.657154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:14.657154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:14.657154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:14.657154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:14.657154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:14.657154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:14.657154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:14.657154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:14.657154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:14.658180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:14.658180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:14.658180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:14.658180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:14.658180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:14.658180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:14.658180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:14.658180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:14.658180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:14.658180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:14.658180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:14.658180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:14.658180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:14.658180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:14.658180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:14.658180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:14.658180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:14.659208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:14.659208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:14.659208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:14.659208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:14.662981 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 324/646] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:14.663614 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06816cd5-47b6-4445-941e-e5e88ef39241 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:14.692636 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c79569-fbf4-4363-8a8d-c3b513fbcc4c None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:14.728354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd3d37-3a50-4864-8659-6f62815b242a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:14.729149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd3d37-3a50-4864-8659-6f62815b242a None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:14.729435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd3d37-3a50-4864-8659-6f62815b242a None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:14.729712 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd3d37-3a50-4864-8659-6f62815b242a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:14.740876 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd3d37-3a50-4864-8659-6f62815b242a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:14.749154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-f6dd3d37-3a50-4864-8659-6f62815b242a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:14.749154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:14.749154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:14.749154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:14.749154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:14.749154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:14.749154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:14.749154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:14.749154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:14.749154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:14.749154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:14.749154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:14.749154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:14.749154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:14.749154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:14.749154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:14.750184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:14.750184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:14.750184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:14.750184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:14.750184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:14.750184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:14.750184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:14.750184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:14.750184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:14.750184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:14.750184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:14.750184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:14.750184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:14.750184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:14.750184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:14.750184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:14.751131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:14.751131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:14.751131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:14.751131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:14.751131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:14.751131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:14.751131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:14.751131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:14.751131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:14.751131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:14.751131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:14.751131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:14.751131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 323/647] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:04:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:14.751131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd3d37-3a50-4864-8659-6f62815b242a None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:14.772658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06816cd5-47b6-4445-941e-e5e88ef39241 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:14.810570 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3287ad-4a2c-4f5d-b0f2-74850a51b6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:14.811554 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3287ad-4a2c-4f5d-b0f2-74850a51b6cd None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:14.811849 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3287ad-4a2c-4f5d-b0f2-74850a51b6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:14.812133 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3287ad-4a2c-4f5d-b0f2-74850a51b6cd None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:14.813010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3287ad-4a2c-4f5d-b0f2-74850a51b6cd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:14.813329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3287ad-4a2c-4f5d-b0f2-74850a51b6cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:14.814762 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3287ad-4a2c-4f5d-b0f2-74850a51b6cd None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:14.836347 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 325/648] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jun 23 09:04:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:14.836347 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3287ad-4a2c-4f5d-b0f2-74850a51b6cd None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:14.863848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:14.867803 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd3d37-3a50-4864-8659-6f62815b242a None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:14.898264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:14.904688 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a02d8b-092e-46fe-a94c-f7283273f24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:14.905356 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a02d8b-092e-46fe-a94c-f7283273f24b None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:14.905644 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a02d8b-092e-46fe-a94c-f7283273f24b None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:14.905928 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a02d8b-092e-46fe-a94c-f7283273f24b None admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:14.906723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a02d8b-092e-46fe-a94c-f7283273f24b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:14.907085 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a02d8b-092e-46fe-a94c-f7283273f24b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:14.908598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a02d8b-092e-46fe-a94c-f7283273f24b None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:14.926139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 324/649] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:14.926139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a02d8b-092e-46fe-a94c-f7283273f24b None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:14.951719 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3287ad-4a2c-4f5d-b0f2-74850a51b6cd None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:14.988990 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d9717-3d4f-432a-8fa8-e612c8765eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:14.989670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d9717-3d4f-432a-8fa8-e612c8765eaf None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:14.989963 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d9717-3d4f-432a-8fa8-e612c8765eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:14.990248 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d9717-3d4f-432a-8fa8-e612c8765eaf None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:14.991008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d9717-3d4f-432a-8fa8-e612c8765eaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:14.991323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d9717-3d4f-432a-8fa8-e612c8765eaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:14.992702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d9717-3d4f-432a-8fa8-e612c8765eaf None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:15.009334 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 326/650] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:15.009334 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d9717-3d4f-432a-8fa8-e612c8765eaf None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:15.033417 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a02d8b-092e-46fe-a94c-f7283273f24b None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:15.069642 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c8f61-90be-43d8-af34-d7cf8b781962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:15.070324 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c8f61-90be-43d8-af34-d7cf8b781962 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:15.070612 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c8f61-90be-43d8-af34-d7cf8b781962 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:15.070888 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c8f61-90be-43d8-af34-d7cf8b781962 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:15.071756 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c8f61-90be-43d8-af34-d7cf8b781962 None admin] RBAC: Adding query filter params (name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:15.072043 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c8f61-90be-43d8-af34-d7cf8b781962 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:15.073422 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c8f61-90be-43d8-af34-d7cf8b781962 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:15.082387 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 325/651] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jun 23 09:04:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:15.087625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c8f61-90be-43d8-af34-d7cf8b781962 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:15.113562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:15.117738 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d9717-3d4f-432a-8fa8-e612c8765eaf None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:15.134437 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:15.156047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47ec5b-7917-4978-a6b9-a91e76a5d6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:15.156777 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47ec5b-7917-4978-a6b9-a91e76a5d6ee None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:15.157135 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47ec5b-7917-4978-a6b9-a91e76a5d6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:15.157501 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47ec5b-7917-4978-a6b9-a91e76a5d6ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:15.198460 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c8f61-90be-43d8-af34-d7cf8b781962 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:15.233367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47ec5b-7917-4978-a6b9-a91e76a5d6ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:15.234527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40225f67-a886-45a9-b0a4-1d3901d68365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:15.235046 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47ec5b-7917-4978-a6b9-a91e76a5d6ee None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:15.235232 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40225f67-a886-45a9-b0a4-1d3901d68365 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:15.235519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40225f67-a886-45a9-b0a4-1d3901d68365 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:15.235800 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40225f67-a886-45a9-b0a4-1d3901d68365 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:15.236632 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40225f67-a886-45a9-b0a4-1d3901d68365 None admin] RBAC: Adding query filter params (role.id=edf696ae763147cab68434524d1eff9c, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=ff025cb706d243778468ae4570374690) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:15.236947 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40225f67-a886-45a9-b0a4-1d3901d68365 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:15.238370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40225f67-a886-45a9-b0a4-1d3901d68365 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:15.244518 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 326/652] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:04:15 2022] GET /identity/v3/role_assignments?user.id=ff025cb706d243778468ae4570374690&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=edf696ae763147cab68434524d1eff9c => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:15.247832 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40225f67-a886-45a9-b0a4-1d3901d68365 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:15.364438 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:15.381249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 327/653] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:04:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:04:15.382083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40225f67-a886-45a9-b0a4-1d3901d68365 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:15.382392 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47ec5b-7917-4978-a6b9-a91e76a5d6ee None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:15.409892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:15.422788 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f219b-89c7-4fba-a620-b553b56d973d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:15.423689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f219b-89c7-4fba-a620-b553b56d973d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:15.423935 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f219b-89c7-4fba-a620-b553b56d973d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:15.426874 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f219b-89c7-4fba-a620-b553b56d973d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:15.432840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f219b-89c7-4fba-a620-b553b56d973d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:15.434412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f219b-89c7-4fba-a620-b553b56d973d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:15.449951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-624f219b-89c7-4fba-a620-b553b56d973d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 09:04:15.449951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:15.449951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:15.449951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:15.449951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:15.449951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:15.449951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:15.449951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:15.449951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:15.449951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:15.449951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:15.449951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:15.449951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:15.449951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:15.449951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:15.449951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:15.451128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:15.451128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:15.451128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:15.451128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:15.451128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:15.451128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:15.451128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:15.451128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:15.451128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:15.451128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:15.451128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:15.451128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:15.451128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:15.451128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:15.451128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:15.451128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:15.451128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:15.451128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:15.452938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:15.452938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:15.452938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:15.452938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:15.452938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:15.452938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:15.452938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:15.452938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:15.452938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 09:04:15.452938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:15.455788 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 327/654] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:04:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:15.456725 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f219b-89c7-4fba-a620-b553b56d973d None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:15.523896 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47ec5b-7917-4978-a6b9-a91e76a5d6ee None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:15.575591 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd33d5-c98f-4074-889c-0d53569ce734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:15.577367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd33d5-c98f-4074-889c-0d53569ce734 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:15.578187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd33d5-c98f-4074-889c-0d53569ce734 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:15.578659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd33d5-c98f-4074-889c-0d53569ce734 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:15.582981 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f219b-89c7-4fba-a620-b553b56d973d None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:15.595670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd33d5-c98f-4074-889c-0d53569ce734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:15.597240 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd33d5-c98f-4074-889c-0d53569ce734 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:15.617622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-0bbd33d5-c98f-4074-889c-0d53569ce734 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:15.617622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:15.617622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:15.617622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:15.617622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:15.617622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:15.617622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:15.617622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:15.617622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:15.617622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:15.617622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:15.617622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:15.617622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:15.617622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:15.617622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:15.618659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:15.618659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:15.618659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:15.618659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:15.618659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:15.618659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:15.618659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:15.618659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:15.618659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:15.618659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:15.618659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:15.618659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:15.618659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:15.618659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:15.618659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:15.618659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:15.618659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:15.619667 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:15.619667 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:15.619667 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:15.619667 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:15.619667 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 328/655] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:15.619957 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd33d5-c98f-4074-889c-0d53569ce734 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:15.637247 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d660a1-67ff-41b2-b0fb-922ce3064ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:15.638312 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d660a1-67ff-41b2-b0fb-922ce3064ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:15.638765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d660a1-67ff-41b2-b0fb-922ce3064ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:15.639211 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d660a1-67ff-41b2-b0fb-922ce3064ba2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:15.651001 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d660a1-67ff-41b2-b0fb-922ce3064ba2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:15.653233 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d660a1-67ff-41b2-b0fb-922ce3064ba2 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:15.660226 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:15.672916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-80d660a1-67ff-41b2-b0fb-922ce3064ba2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:15.672916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:15.672916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:15.672916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:15.672916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:15.672916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:15.672916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:15.672916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:15.672916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:15.672916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:15.672916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:15.672916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:15.672916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:15.672916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:15.672916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:15.678524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:15.678524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:15.678524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:15.678524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:15.678524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:15.678524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:15.678524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:15.678524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:15.678524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:15.678524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:15.678524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:15.678524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:15.678524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:15.678524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:15.678524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:15.678524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:15.678524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:15.679619 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:15.679619 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:15.679619 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:15.679619 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:15.679619 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 328/656] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:15.683576 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d660a1-67ff-41b2-b0fb-922ce3064ba2 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:15.700601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:15.742731 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd33d5-c98f-4074-889c-0d53569ce734 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:15.780499 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4889b5-fc73-4272-817c-eca3652f995e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:15.781231 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4889b5-fc73-4272-817c-eca3652f995e None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:15.781527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4889b5-fc73-4272-817c-eca3652f995e None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:15.781809 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4889b5-fc73-4272-817c-eca3652f995e None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:15.782700 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4889b5-fc73-4272-817c-eca3652f995e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:15.783052 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4889b5-fc73-4272-817c-eca3652f995e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:15.784669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4889b5-fc73-4272-817c-eca3652f995e None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:15.812255 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d660a1-67ff-41b2-b0fb-922ce3064ba2 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:15.813235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 329/657] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jun 23 09:04:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:15.814182 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4889b5-fc73-4272-817c-eca3652f995e None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:15.849721 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2b4ee-8b42-4f3a-8591-911d8f93533d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:15.850679 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2b4ee-8b42-4f3a-8591-911d8f93533d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:15.851147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2b4ee-8b42-4f3a-8591-911d8f93533d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:15.851568 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2b4ee-8b42-4f3a-8591-911d8f93533d None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:15.852458 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2b4ee-8b42-4f3a-8591-911d8f93533d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:15.852911 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2b4ee-8b42-4f3a-8591-911d8f93533d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:15.854444 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2b4ee-8b42-4f3a-8591-911d8f93533d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:15.868181 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 329/658] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:15.869243 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2b4ee-8b42-4f3a-8591-911d8f93533d None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:15.927871 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4889b5-fc73-4272-817c-eca3652f995e None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:15.966703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:15.969093 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:15.977777 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f10517-2c97-4146-ad34-ccac0df39bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:15.978411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f10517-2c97-4146-ad34-ccac0df39bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:15.978701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f10517-2c97-4146-ad34-ccac0df39bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:15.979008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f10517-2c97-4146-ad34-ccac0df39bc4 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:15.979749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f10517-2c97-4146-ad34-ccac0df39bc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:15.980061 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f10517-2c97-4146-ad34-ccac0df39bc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:15.981507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f10517-2c97-4146-ad34-ccac0df39bc4 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:15.993772 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 330/659] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:15.994721 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f10517-2c97-4146-ad34-ccac0df39bc4 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:16.022369 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2b4ee-8b42-4f3a-8591-911d8f93533d None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:16.065067 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e7c21-edd0-49e0-8d85-c66f66784462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:16.065878 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e7c21-edd0-49e0-8d85-c66f66784462 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:16.066257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e7c21-edd0-49e0-8d85-c66f66784462 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:16.066628 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e7c21-edd0-49e0-8d85-c66f66784462 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:16.092843 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f10517-2c97-4146-ad34-ccac0df39bc4 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:16.128248 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f867e2-9fe8-4f2c-9e30-6364bc5a0321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:16.129061 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f867e2-9fe8-4f2c-9e30-6364bc5a0321 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:16.129350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f867e2-9fe8-4f2c-9e30-6364bc5a0321 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:16.129641 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f867e2-9fe8-4f2c-9e30-6364bc5a0321 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:16.139494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f867e2-9fe8-4f2c-9e30-6364bc5a0321 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:16.141390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e7c21-edd0-49e0-8d85-c66f66784462 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:16.141508 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f867e2-9fe8-4f2c-9e30-6364bc5a0321 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:16.142471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e7c21-edd0-49e0-8d85-c66f66784462 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:16.156254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-f1f867e2-9fe8-4f2c-9e30-6364bc5a0321 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 09:04:16.156254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:16.156254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:16.156254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:16.156254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:16.156254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:16.156254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:16.156254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:16.156254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:16.156254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:16.156254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:16.156254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:16.156254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:16.156254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:16.156254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:16.156254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:16.157277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:16.157277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:16.157277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:16.157277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:16.157277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:16.157277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:16.157277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:16.157277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:16.157277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:16.157277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:16.157277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:16.157277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:16.157277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:16.157277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:16.157277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:16.157277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:16.157277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:16.157277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:16.158283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:16.158283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:16.158283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:16.158283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:16.158283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:16.158283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:16.158283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:16.158283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:16.158283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 09:04:16.158283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:16.158283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 331/660] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:04:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:16.159679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef79e9-f7ba-47e5-a156-4df89195ee3e None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:16.159679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef79e9-f7ba-47e5-a156-4df89195ee3e None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:16.159679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef79e9-f7ba-47e5-a156-4df89195ee3e None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:16.160845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 332/661] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:16.165588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef79e9-f7ba-47e5-a156-4df89195ee3e None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:16.212918 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:16.219258 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:16.260973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 330/662] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:04:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:04:16.260973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e7c21-edd0-49e0-8d85-c66f66784462 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:16.275630 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef79e9-f7ba-47e5-a156-4df89195ee3e None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:16.318559 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843dcdac-e3f4-4b7d-af50-fe458d9377b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:16.319392 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dcdac-e3f4-4b7d-af50-fe458d9377b2 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:16.319676 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dcdac-e3f4-4b7d-af50-fe458d9377b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:16.319949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dcdac-e3f4-4b7d-af50-fe458d9377b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:16.330529 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843dcdac-e3f4-4b7d-af50-fe458d9377b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:16.331929 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843dcdac-e3f4-4b7d-af50-fe458d9377b2 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:16.346859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-843dcdac-e3f4-4b7d-af50-fe458d9377b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:16.346859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:16.346859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:16.346859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:16.346859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:16.346859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:16.346859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:16.346859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:16.346859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:16.346859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:16.346859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:16.346859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:16.346859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:16.346859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:16.346859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:16.347842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:16.347842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:16.347842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:16.347842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:16.347842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:16.347842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:16.347842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:16.347842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:16.347842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:16.347842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:16.347842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:16.347842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:16.347842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:16.347842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:16.347842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:16.347842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:16.347842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:16.348887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:16.348887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:16.348887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:16.348887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:16.348887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:16.348887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:16.348887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:16.348887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:16.352732 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 333/663] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:16.353975 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607f14b-b999-4cf2-a6dd-f0520ff1de9e None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:16.354263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607f14b-b999-4cf2-a6dd-f0520ff1de9e None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:16.354536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607f14b-b999-4cf2-a6dd-f0520ff1de9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:16.362880 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e7c21-edd0-49e0-8d85-c66f66784462 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:16.372805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-b607f14b-b999-4cf2-a6dd-f0520ff1de9e None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:16.373993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b607f14b-b999-4cf2-a6dd-f0520ff1de9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:16.400301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59dfe39-e27e-47a1-9971-824a26e65ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:16.400968 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59dfe39-e27e-47a1-9971-824a26e65ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:16.401258 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59dfe39-e27e-47a1-9971-824a26e65ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:16.401529 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59dfe39-e27e-47a1-9971-824a26e65ba2 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:16.402390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59dfe39-e27e-47a1-9971-824a26e65ba2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:16.402705 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59dfe39-e27e-47a1-9971-824a26e65ba2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:16.404184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59dfe39-e27e-47a1-9971-824a26e65ba2 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:16.421118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 331/664] 10.222.0.52 () {56 vars in 1139 bytes} [Thu Jun 23 09:04:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:16.421921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59dfe39-e27e-47a1-9971-824a26e65ba2 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:16.441866 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:16.448914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 334/665] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:04:16.450558 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607f14b-b999-4cf2-a6dd-f0520ff1de9e None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:16.496997 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:16.539334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59dfe39-e27e-47a1-9971-824a26e65ba2 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:16.568010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607f14b-b999-4cf2-a6dd-f0520ff1de9e None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:16.573726 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a1c15-f9c7-4a09-9efd-46a5238ba01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:16.575072 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a1c15-f9c7-4a09-9efd-46a5238ba01e None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:16.575358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a1c15-f9c7-4a09-9efd-46a5238ba01e None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:16.575628 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a1c15-f9c7-4a09-9efd-46a5238ba01e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:16.586074 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a1c15-f9c7-4a09-9efd-46a5238ba01e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:16.587545 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a1c15-f9c7-4a09-9efd-46a5238ba01e None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:16.604170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f8c662-d8bd-4536-bcee-2686c68a3abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:16.604921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8c662-d8bd-4536-bcee-2686c68a3abf None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:16.605210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8c662-d8bd-4536-bcee-2686c68a3abf None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:16.606013 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8c662-d8bd-4536-bcee-2686c68a3abf None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:16.606858 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f8c662-d8bd-4536-bcee-2686c68a3abf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:16.607207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f8c662-d8bd-4536-bcee-2686c68a3abf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:16.607305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-fc5a1c15-f9c7-4a09-9efd-46a5238ba01e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:16.607305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:16.607305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:16.607305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:16.607305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:16.607305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:16.607305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:16.607305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:16.607305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:16.607305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:16.607305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:16.607305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:16.607305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:16.607305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:16.607305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:16.608182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:16.608182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:16.608182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:16.608182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:16.608182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:16.608182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:16.608182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:16.608182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:16.608182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:16.608182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:16.608182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:16.608182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:16.608182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:16.608182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:16.608182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:16.608182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:16.608182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:16.609132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:16.609132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:16.609132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:16.609132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:16.609381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f8c662-d8bd-4536-bcee-2686c68a3abf None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:16.613249 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 332/666] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:16.613976 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a1c15-f9c7-4a09-9efd-46a5238ba01e None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:16.625412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 335/667] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:16.625412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f8c662-d8bd-4536-bcee-2686c68a3abf None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:16.686552 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:16.720199 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a1c15-f9c7-4a09-9efd-46a5238ba01e None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:16.730370 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:16.736357 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f8c662-d8bd-4536-bcee-2686c68a3abf None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:16.758140 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1b731-b211-4624-899d-c165d51c2160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:16.759255 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1b731-b211-4624-899d-c165d51c2160 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:16.759552 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1b731-b211-4624-899d-c165d51c2160 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:16.760091 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1b731-b211-4624-899d-c165d51c2160 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:16.770004 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6329f-73f2-442a-8926-aa4330b8ce5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:16.770793 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6329f-73f2-442a-8926-aa4330b8ce5c None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:16.771107 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6329f-73f2-442a-8926-aa4330b8ce5c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:16.771386 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6329f-73f2-442a-8926-aa4330b8ce5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:16.782474 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6329f-73f2-442a-8926-aa4330b8ce5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:16.790803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-3cc6329f-73f2-442a-8926-aa4330b8ce5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:16.790803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:16.790803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:16.790803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:16.790803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:16.790803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:16.790803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:16.790803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:16.790803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:16.790803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:16.790803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:16.790803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:16.790803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:16.790803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:16.790803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:16.790803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:16.791921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:16.791921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:16.791921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:16.791921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:16.791921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:16.791921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:16.791921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:16.791921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:16.791921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:16.791921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:16.791921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:16.791921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:16.791921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:16.791921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:16.791921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:16.791921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:16.792830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:16.792830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:16.792830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:16.792830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:16.792830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:16.792830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:16.792830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:16.792830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:16.792830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:16.792830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:16.792830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:16.792830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:16.792830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 336/668] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:04:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:16.793631 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6329f-73f2-442a-8926-aa4330b8ce5c None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:16.838549 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1b731-b211-4624-899d-c165d51c2160 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:16.840450 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1b731-b211-4624-899d-c165d51c2160 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:16.898284 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6329f-73f2-442a-8926-aa4330b8ce5c None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:16.912731 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:16.935991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f46591-8286-4c6b-9246-b01bce7aa347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:16.936671 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f46591-8286-4c6b-9246-b01bce7aa347 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:16.936962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f46591-8286-4c6b-9246-b01bce7aa347 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:16.937240 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f46591-8286-4c6b-9246-b01bce7aa347 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:16.937969 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f46591-8286-4c6b-9246-b01bce7aa347 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:16.938298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f46591-8286-4c6b-9246-b01bce7aa347 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:16.939718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f46591-8286-4c6b-9246-b01bce7aa347 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:16.952631 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 333/669] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:04:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:04:16.952631 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1b731-b211-4624-899d-c165d51c2160 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:16.956536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 337/670] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:16.961228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f46591-8286-4c6b-9246-b01bce7aa347 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:16.980246 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:17.055043 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1b731-b211-4624-899d-c165d51c2160 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:17.066597 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f46591-8286-4c6b-9246-b01bce7aa347 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:17.093427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad5436-c69d-41e9-9d95-cb4a84c118ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:17.094096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad5436-c69d-41e9-9d95-cb4a84c118ff None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:17.094383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad5436-c69d-41e9-9d95-cb4a84c118ff None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:17.094664 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad5436-c69d-41e9-9d95-cb4a84c118ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:17.095521 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad5436-c69d-41e9-9d95-cb4a84c118ff None admin] RBAC: Adding query filter params (role.id=edf696ae763147cab68434524d1eff9c, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=36a6a3190dad4948a505bbe4c7095699) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:17.095832 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad5436-c69d-41e9-9d95-cb4a84c118ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:17.097350 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad5436-c69d-41e9-9d95-cb4a84c118ff None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:17.102290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc44c9-b10f-4967-b8d2-ff23d7ca2236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:17.102998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc44c9-b10f-4967-b8d2-ff23d7ca2236 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:17.103292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc44c9-b10f-4967-b8d2-ff23d7ca2236 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:17.113650 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc44c9-b10f-4967-b8d2-ff23d7ca2236 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:17.113870 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 334/671] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:04:16 2022] GET /identity/v3/role_assignments?user.id=36a6a3190dad4948a505bbe4c7095699&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=edf696ae763147cab68434524d1eff9c => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:17.113993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc44c9-b10f-4967-b8d2-ff23d7ca2236 None admin] RBAC: Adding query filter params (name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:17.113993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc44c9-b10f-4967-b8d2-ff23d7ca2236 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:17.114159 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad5436-c69d-41e9-9d95-cb4a84c118ff None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:17.114245 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc44c9-b10f-4967-b8d2-ff23d7ca2236 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:17.115294 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 338/672] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jun 23 09:04:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:17.116239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc44c9-b10f-4967-b8d2-ff23d7ca2236 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:17.156872 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:17.234748 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:17.249247 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad5436-c69d-41e9-9d95-cb4a84c118ff None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:17.255531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc44c9-b10f-4967-b8d2-ff23d7ca2236 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:17.294301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5728c-b8dd-4eb4-9620-741336aad3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:17.295147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5728c-b8dd-4eb4-9620-741336aad3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:17.295439 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5728c-b8dd-4eb4-9620-741336aad3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:17.295721 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5728c-b8dd-4eb4-9620-741336aad3f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:17.303177 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2ab2e-2815-474d-922a-bb2b57729ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:17.303979 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2ab2e-2815-474d-922a-bb2b57729ffb None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:17.304264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2ab2e-2815-474d-922a-bb2b57729ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:17.304542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2ab2e-2815-474d-922a-bb2b57729ffb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:17.314069 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5728c-b8dd-4eb4-9620-741336aad3f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:17.314069 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5728c-b8dd-4eb4-9620-741336aad3f0 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:17.316596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a2ab2e-2815-474d-922a-bb2b57729ffb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:17.318053 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a2ab2e-2815-474d-922a-bb2b57729ffb None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:17.332400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-f2f5728c-b8dd-4eb4-9620-741336aad3f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:17.332400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:17.332400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:17.332400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:17.332400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:17.332400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:17.332400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:17.332400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:17.332400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:17.332400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:17.332400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:17.332400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:17.332400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:17.332400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:17.332400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:17.333848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:17.333848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:17.333848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:17.333848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:17.333848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:17.333848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:17.333848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:17.333848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:17.333848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:17.333848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:17.333848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:17.333848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:17.333848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:17.333848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:17.333848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:17.333848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:17.333848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:17.335533 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:17.335533 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:17.335533 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:17.335533 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:17.338635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 335/673] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:17.338756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-b9a2ab2e-2815-474d-922a-bb2b57729ffb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:17.338756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:17.338756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:17.338756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:17.338756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:17.338756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:17.338756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:17.338756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:17.338756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:17.338756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:17.338756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:17.338756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:17.338756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:17.338756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:17.338756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:17.339670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:17.339670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:17.339670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:17.339670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:17.339670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:17.339670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:17.339670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:17.339670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:17.339670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:17.339670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:17.339670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:17.339670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:17.339670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:17.339670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:17.339670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:17.339670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:17.339670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:17.342483 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:17.342483 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:17.342483 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:17.342483 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:17.342483 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:17.342483 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:17.342483 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:17.342483 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:17.343536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5728c-b8dd-4eb4-9620-741336aad3f0 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:17.343683 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 339/674] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:17.343683 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2ab2e-2815-474d-922a-bb2b57729ffb None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:17.443038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:17.463242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2ab2e-2815-474d-922a-bb2b57729ffb None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:17.465795 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5728c-b8dd-4eb4-9620-741336aad3f0 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:17.502056 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd4e8f-765b-418f-b158-3d32f8a48ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:17.503071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd4e8f-765b-418f-b158-3d32f8a48ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:17.503567 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd4e8f-765b-418f-b158-3d32f8a48ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:17.504027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd4e8f-765b-418f-b158-3d32f8a48ed6 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:17.504933 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd4e8f-765b-418f-b158-3d32f8a48ed6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:17.505419 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd4e8f-765b-418f-b158-3d32f8a48ed6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:17.506982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd4e8f-765b-418f-b158-3d32f8a48ed6 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:17.507316 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8357d9-f7f7-4679-8b70-d114e1f3a93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:17.508117 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8357d9-f7f7-4679-8b70-d114e1f3a93c None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:17.508403 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8357d9-f7f7-4679-8b70-d114e1f3a93c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:17.508686 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8357d9-f7f7-4679-8b70-d114e1f3a93c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:17.516658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:17.519807 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8357d9-f7f7-4679-8b70-d114e1f3a93c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:17.521263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8357d9-f7f7-4679-8b70-d114e1f3a93c None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:17.525949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 340/675] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:17.525949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd4e8f-765b-418f-b158-3d32f8a48ed6 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:17.544792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-dd8357d9-f7f7-4679-8b70-d114e1f3a93c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:17.544792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:17.544792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:17.544792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:17.544792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:17.544792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:17.544792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:17.544792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:17.544792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:17.544792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:17.544792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:17.544792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:17.544792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:17.544792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:17.544792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:17.545982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:17.545982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:17.545982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:17.545982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:17.545982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:17.545982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:17.545982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:17.545982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:17.545982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:17.545982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:17.545982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:17.545982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:17.545982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:17.545982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:17.545982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:17.545982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:17.545982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:17.547770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:17.547770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:17.547770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:17.547770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:17.547770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 336/676] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:17.547770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5aeb3-32e9-43bc-b370-ee2e26fdb578 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:17.547770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5aeb3-32e9-43bc-b370-ee2e26fdb578 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:17.548213 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5aeb3-32e9-43bc-b370-ee2e26fdb578 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:17.549420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 337/677] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:17.550142 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5aeb3-32e9-43bc-b370-ee2e26fdb578 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:17.631312 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd4e8f-765b-418f-b158-3d32f8a48ed6 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:17.665159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202918e-6737-44d9-ba4b-a9cf46152b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:17.665831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202918e-6737-44d9-ba4b-a9cf46152b79 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:17.666120 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202918e-6737-44d9-ba4b-a9cf46152b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:17.666398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202918e-6737-44d9-ba4b-a9cf46152b79 None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:17.667227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202918e-6737-44d9-ba4b-a9cf46152b79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:17.667540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202918e-6737-44d9-ba4b-a9cf46152b79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:17.669045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202918e-6737-44d9-ba4b-a9cf46152b79 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:17.669649 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5aeb3-32e9-43bc-b370-ee2e26fdb578 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:17.680609 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 341/678] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:17.681412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202918e-6737-44d9-ba4b-a9cf46152b79 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:17.714192 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ca8ec-0299-4a20-a3a2-56e8961b5ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:17.715455 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ca8ec-0299-4a20-a3a2-56e8961b5ddf None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:17.715950 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ca8ec-0299-4a20-a3a2-56e8961b5ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:17.716458 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ca8ec-0299-4a20-a3a2-56e8961b5ddf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:17.722448 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:17.726868 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ca8ec-0299-4a20-a3a2-56e8961b5ddf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:17.728342 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ca8ec-0299-4a20-a3a2-56e8961b5ddf None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:17.742251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-b41ca8ec-0299-4a20-a3a2-56e8961b5ddf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:17.742251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:17.742251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:17.742251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:17.742251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:17.742251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:17.742251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:17.742251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:17.742251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:17.742251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:17.742251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:17.742251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:17.742251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:17.742251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:17.742251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:17.749179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:17.749179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:17.749179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:17.749179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:17.749179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:17.749179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:17.749179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:17.749179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:17.749179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:17.749179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:17.749179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:17.749179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:17.749179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:17.749179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:17.749179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:17.749179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:17.749179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:17.750382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:17.750382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:17.750382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:17.750382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:17.750382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:17.750382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:17.750382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:17.750382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:17.750382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 338/679] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:17.750382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a628356-d435-4b48-ab79-05985c6e793c None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:17.750382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a628356-d435-4b48-ab79-05985c6e793c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:17.751233 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a628356-d435-4b48-ab79-05985c6e793c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:17.756638 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:17.767567 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-8a628356-d435-4b48-ab79-05985c6e793c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:17.768829 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-8a628356-d435-4b48-ab79-05985c6e793c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:17.799724 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202918e-6737-44d9-ba4b-a9cf46152b79 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:17.840624 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb1803-e8df-4ee6-b0a9-f12393ee4bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:17.841299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb1803-e8df-4ee6-b0a9-f12393ee4bdd None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:17.841588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb1803-e8df-4ee6-b0a9-f12393ee4bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:17.841866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb1803-e8df-4ee6-b0a9-f12393ee4bdd None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:17.842594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb1803-e8df-4ee6-b0a9-f12393ee4bdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:17.842898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb1803-e8df-4ee6-b0a9-f12393ee4bdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:17.843728 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 339/680] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:04:17.844421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a628356-d435-4b48-ab79-05985c6e793c None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:17.853273 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb1803-e8df-4ee6-b0a9-f12393ee4bdd None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:17.869802 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 342/681] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:17.870761 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb1803-e8df-4ee6-b0a9-f12393ee4bdd None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:17.952843 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a628356-d435-4b48-ab79-05985c6e793c None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:17.976713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb1803-e8df-4ee6-b0a9-f12393ee4bdd None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:17.982084 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:17.990867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72890e6-0a1a-4a17-abe1-392cfb40692e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:17.995977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72890e6-0a1a-4a17-abe1-392cfb40692e None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:17.995977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72890e6-0a1a-4a17-abe1-392cfb40692e None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:17.995977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72890e6-0a1a-4a17-abe1-392cfb40692e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:17.995977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72890e6-0a1a-4a17-abe1-392cfb40692e None admin] RBAC: Adding query filter params (role.id=edf696ae763147cab68434524d1eff9c, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=c21a758d0e2645318c89a4fe8805df0a) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:17.995977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72890e6-0a1a-4a17-abe1-392cfb40692e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:17.995977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72890e6-0a1a-4a17-abe1-392cfb40692e None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:17.996448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:18.001310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 340/682] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:04:17 2022] GET /identity/v3/role_assignments?user.id=c21a758d0e2645318c89a4fe8805df0a&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=edf696ae763147cab68434524d1eff9c => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:18.001891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72890e6-0a1a-4a17-abe1-392cfb40692e None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:18.024259 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b4022-55fd-4850-90b6-fff8aaf9c9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:18.024958 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b4022-55fd-4850-90b6-fff8aaf9c9a2 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:18.025244 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b4022-55fd-4850-90b6-fff8aaf9c9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:18.025522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b4022-55fd-4850-90b6-fff8aaf9c9a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:18.026414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b4022-55fd-4850-90b6-fff8aaf9c9a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:18.026730 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b4022-55fd-4850-90b6-fff8aaf9c9a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:18.028701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b4022-55fd-4850-90b6-fff8aaf9c9a2 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:18.044981 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 343/683] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:18.045943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b4022-55fd-4850-90b6-fff8aaf9c9a2 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:18.135291 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72890e6-0a1a-4a17-abe1-392cfb40692e None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:18.173979 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04179a8-7aa1-46bb-9329-5f474a2be0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:18.174796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04179a8-7aa1-46bb-9329-5f474a2be0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:18.175208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04179a8-7aa1-46bb-9329-5f474a2be0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:18.175319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b4022-55fd-4850-90b6-fff8aaf9c9a2 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:18.175485 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04179a8-7aa1-46bb-9329-5f474a2be0c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:18.189120 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04179a8-7aa1-46bb-9329-5f474a2be0c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:18.197141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-d04179a8-7aa1-46bb-9329-5f474a2be0c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:04:18.197141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:18.197141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:18.197141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:18.197141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:18.197141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:18.197141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:18.197141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:18.197141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:18.197141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:18.197141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:18.197141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:18.197141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:18.197141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:18.197141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:18.197141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:18.198127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:18.198127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:18.198127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:18.198127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:18.198127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:18.198127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:18.198127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:18.198127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:18.198127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:18.198127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:18.198127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:18.198127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:18.198127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:18.198127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:18.198127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:18.198127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:18.199070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:18.199070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:18.199070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:18.199070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:18.199070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:18.199070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:18.199070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:18.199070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:18.199070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:18.199070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:18.199070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:04:18.199070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:18.201528 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 341/684] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:04:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:18.202578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04179a8-7aa1-46bb-9329-5f474a2be0c9 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:18.215648 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b68d3a-340e-460c-9560-b583e413645e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:18.216865 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b68d3a-340e-460c-9560-b583e413645e None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:18.217160 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b68d3a-340e-460c-9560-b583e413645e None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:18.217444 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b68d3a-340e-460c-9560-b583e413645e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:18.227552 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b68d3a-340e-460c-9560-b583e413645e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:18.229468 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b68d3a-340e-460c-9560-b583e413645e None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:18.247299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-42b68d3a-340e-460c-9560-b583e413645e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 09:04:18.247299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:18.247299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:18.247299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:18.247299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:18.247299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:18.247299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:18.247299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:18.247299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:18.247299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:18.247299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:18.247299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:18.247299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:18.247299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:18.247299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:18.247299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:18.248329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:18.248329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:18.248329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:18.248329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:18.248329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:18.248329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:18.248329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:18.248329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:18.248329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:18.248329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:18.248329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:18.248329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:18.248329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:18.248329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:18.248329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:18.248329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:18.248329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:18.248329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:18.249362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:18.249362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:18.249362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:18.249362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:18.249362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:18.249362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:18.249362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:18.249362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:18.249362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 09:04:18.249362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:18.249362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 344/685] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:04:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:18.250000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b68d3a-340e-460c-9560-b583e413645e None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:18.273386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:18.295355 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:18.341679 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04179a8-7aa1-46bb-9329-5f474a2be0c9 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:18.364030 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b68d3a-340e-460c-9560-b583e413645e None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:18.384831 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c8930-5ca5-4453-8205-9262ec9accbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:18.385766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c8930-5ca5-4453-8205-9262ec9accbc None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:18.386222 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c8930-5ca5-4453-8205-9262ec9accbc None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:18.386678 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c8930-5ca5-4453-8205-9262ec9accbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:18.388131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c8930-5ca5-4453-8205-9262ec9accbc None admin] RBAC: Adding query filter params (role.id=edf696ae763147cab68434524d1eff9c, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=daed809f511a43ed8d3f88eba10466b2) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:18.388608 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c8930-5ca5-4453-8205-9262ec9accbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:18.390683 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c8930-5ca5-4453-8205-9262ec9accbc None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:18.404473 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 342/686] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:04:18 2022] GET /identity/v3/role_assignments?user.id=daed809f511a43ed8d3f88eba10466b2&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=edf696ae763147cab68434524d1eff9c => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:18.404473 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c8930-5ca5-4453-8205-9262ec9accbc None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:18.422928 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc698661-09c5-4f27-9aca-77d5b736456b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:18.423710 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc698661-09c5-4f27-9aca-77d5b736456b None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:18.424051 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc698661-09c5-4f27-9aca-77d5b736456b None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:18.424335 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc698661-09c5-4f27-9aca-77d5b736456b None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:18.425186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc698661-09c5-4f27-9aca-77d5b736456b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:18.425503 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc698661-09c5-4f27-9aca-77d5b736456b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:18.427038 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc698661-09c5-4f27-9aca-77d5b736456b None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:18.447341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 345/687] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:04:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:18.452763 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc698661-09c5-4f27-9aca-77d5b736456b None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:18.600437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c8930-5ca5-4453-8205-9262ec9accbc None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:18.610611 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc698661-09c5-4f27-9aca-77d5b736456b None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:18.620954 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:18.623295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:18.646732 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa85500-b433-4fda-b13c-8a1d5a2e5287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:18.647451 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa85500-b433-4fda-b13c-8a1d5a2e5287 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:18.647737 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa85500-b433-4fda-b13c-8a1d5a2e5287 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:18.648043 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa85500-b433-4fda-b13c-8a1d5a2e5287 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:18.648854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa85500-b433-4fda-b13c-8a1d5a2e5287 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:18.649162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa85500-b433-4fda-b13c-8a1d5a2e5287 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:18.650606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa85500-b433-4fda-b13c-8a1d5a2e5287 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:18.651601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f807cf-a58c-48f3-8b15-26d1e2fb8aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:18.653166 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f807cf-a58c-48f3-8b15-26d1e2fb8aeb None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:18.653657 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f807cf-a58c-48f3-8b15-26d1e2fb8aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:18.654149 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f807cf-a58c-48f3-8b15-26d1e2fb8aeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:18.666486 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f807cf-a58c-48f3-8b15-26d1e2fb8aeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:18.668442 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f807cf-a58c-48f3-8b15-26d1e2fb8aeb None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:18.670357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 343/688] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jun 23 09:04:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:18.675766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa85500-b433-4fda-b13c-8a1d5a2e5287 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:18.688770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-45f807cf-a58c-48f3-8b15-26d1e2fb8aeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:18.688770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:18.688770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:18.688770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:18.688770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:18.688770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:18.688770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:18.688770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:18.688770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:18.688770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:18.688770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:18.688770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:18.688770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:18.688770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:18.688770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:18.689745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:18.689745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:18.689745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:18.689745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:18.689745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:18.689745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:18.689745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:18.689745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:18.689745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:18.689745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:18.689745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:18.689745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:18.689745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:18.689745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:18.689745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:18.689745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:18.689745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:18.690713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:18.690713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:18.690713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:18.690713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:18.693070 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 346/689] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:18.699874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f807cf-a58c-48f3-8b15-26d1e2fb8aeb None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:18.793556 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa85500-b433-4fda-b13c-8a1d5a2e5287 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:18.814105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f807cf-a58c-48f3-8b15-26d1e2fb8aeb None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:18.834297 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50649d-58e6-4393-81e1-9c67b98b532d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:18.834981 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50649d-58e6-4393-81e1-9c67b98b532d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:18.835269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50649d-58e6-4393-81e1-9c67b98b532d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:18.835542 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50649d-58e6-4393-81e1-9c67b98b532d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:18.854069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0212d78-3183-4467-8ae4-07a61f1e57e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:18.855191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0212d78-3183-4467-8ae4-07a61f1e57e9 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:18.855746 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0212d78-3183-4467-8ae4-07a61f1e57e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:18.856232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0212d78-3183-4467-8ae4-07a61f1e57e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:18.857307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0212d78-3183-4467-8ae4-07a61f1e57e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:18.857871 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0212d78-3183-4467-8ae4-07a61f1e57e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:18.859601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0212d78-3183-4467-8ae4-07a61f1e57e9 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:18.873301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 347/690] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:18.879394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0212d78-3183-4467-8ae4-07a61f1e57e9 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:18.899083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:18.915300 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:18.960760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50649d-58e6-4393-81e1-9c67b98b532d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:18.962239 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50649d-58e6-4393-81e1-9c67b98b532d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:19.038279 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0212d78-3183-4467-8ae4-07a61f1e57e9 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:19.075423 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd414bf-f3c5-4bdf-92d9-9236b577524d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:19.076234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd414bf-f3c5-4bdf-92d9-9236b577524d None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:19.076534 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd414bf-f3c5-4bdf-92d9-9236b577524d None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:19.076817 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd414bf-f3c5-4bdf-92d9-9236b577524d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:19.084450 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd414bf-f3c5-4bdf-92d9-9236b577524d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:19.085912 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd414bf-f3c5-4bdf-92d9-9236b577524d None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:19.102217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-7fd414bf-f3c5-4bdf-92d9-9236b577524d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 09:04:19.102217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:19.102217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:19.102217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:19.102217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:19.102217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:19.102217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:19.102217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:19.102217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:19.102217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:19.102217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:19.102217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:19.102217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:19.102217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:19.102217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:19.102217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:19.103281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:19.103281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:19.103281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:19.103281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:19.103281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:19.103281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:19.103281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:19.103281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:19.103281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:19.103281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:19.103281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:19.103281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:19.103281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:19.103281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:19.103281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:19.103281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:19.103281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:19.103281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:19.104411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:19.104411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:19.104411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:19.104411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:19.104411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:19.104411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:19.104411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:19.104411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:19.104411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 09:04:19.104411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:19.105018 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 348/691] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jun 23 09:04:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:19.111866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd414bf-f3c5-4bdf-92d9-9236b577524d None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:19.149654 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 344/692] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:04:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:04:19.160305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50649d-58e6-4393-81e1-9c67b98b532d None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:19.215913 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:19.261673 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd414bf-f3c5-4bdf-92d9-9236b577524d None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:19.302313 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:19.312825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee0a5e-cac6-4e41-bb1e-0727d9e25288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:19.313819 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee0a5e-cac6-4e41-bb1e-0727d9e25288 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:19.314281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee0a5e-cac6-4e41-bb1e-0727d9e25288 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:19.314741 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee0a5e-cac6-4e41-bb1e-0727d9e25288 None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:19.315804 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee0a5e-cac6-4e41-bb1e-0727d9e25288 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:19.316301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee0a5e-cac6-4e41-bb1e-0727d9e25288 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:19.317973 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee0a5e-cac6-4e41-bb1e-0727d9e25288 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:19.337697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50649d-58e6-4393-81e1-9c67b98b532d None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:19.338736 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 349/693] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jun 23 09:04:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:19.352021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee0a5e-cac6-4e41-bb1e-0727d9e25288 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:19.387408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951bc0f-0257-4ccd-847d-d3c70b1f8f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:19.387408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951bc0f-0257-4ccd-847d-d3c70b1f8f3e None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:19.387408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951bc0f-0257-4ccd-847d-d3c70b1f8f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:19.387408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951bc0f-0257-4ccd-847d-d3c70b1f8f3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:19.391841 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951bc0f-0257-4ccd-847d-d3c70b1f8f3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:19.393395 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951bc0f-0257-4ccd-847d-d3c70b1f8f3e None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:19.412891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-4951bc0f-0257-4ccd-847d-d3c70b1f8f3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:19.412891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:19.412891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:19.412891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:19.412891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:19.412891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:19.412891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:19.412891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:19.412891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:19.412891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:19.412891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:19.412891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:19.412891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:19.412891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:19.412891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:19.414093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:19.414093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:19.414093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:19.414093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:19.414093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:19.414093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:19.414093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:19.414093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:19.414093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:19.414093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:19.414093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:19.414093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:19.414093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:19.414093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:19.414093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:19.414093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:19.414093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:19.415142 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:19.415142 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:19.415142 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:19.415142 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:19.415142 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 345/694] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:19.422935 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951bc0f-0257-4ccd-847d-d3c70b1f8f3e None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:19.491148 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee0a5e-cac6-4e41-bb1e-0727d9e25288 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:19.529774 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951bc0f-0257-4ccd-847d-d3c70b1f8f3e None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:19.539563 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:19.546717 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debee408-5107-46dc-b51c-feacfc6a08eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:19.547685 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debee408-5107-46dc-b51c-feacfc6a08eb None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:19.548186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debee408-5107-46dc-b51c-feacfc6a08eb None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:19.548710 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debee408-5107-46dc-b51c-feacfc6a08eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:19.559357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:19.566189 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820cb41-9736-4802-906e-25b81c86f334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:19.566960 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820cb41-9736-4802-906e-25b81c86f334 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:19.592369 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820cb41-9736-4802-906e-25b81c86f334 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:19.592369 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820cb41-9736-4802-906e-25b81c86f334 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:19.592369 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820cb41-9736-4802-906e-25b81c86f334 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:19.592369 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820cb41-9736-4802-906e-25b81c86f334 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:19.592369 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820cb41-9736-4802-906e-25b81c86f334 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:19.592369 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 346/695] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:19.592949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a2131-f80a-4dc3-aedf-3068e7d1ee3d None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:19.592949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a2131-f80a-4dc3-aedf-3068e7d1ee3d None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:19.592949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a2131-f80a-4dc3-aedf-3068e7d1ee3d None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:19.592949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 347/696] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:19.592949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a2131-f80a-4dc3-aedf-3068e7d1ee3d None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:19.626030 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debee408-5107-46dc-b51c-feacfc6a08eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:19.627738 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debee408-5107-46dc-b51c-feacfc6a08eb None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:19.695838 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a2131-f80a-4dc3-aedf-3068e7d1ee3d None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:19.731256 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5aca5-f882-4782-a116-577a73b931de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:19.732147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5aca5-f882-4782-a116-577a73b931de None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:19.732507 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5aca5-f882-4782-a116-577a73b931de None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:19.732853 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5aca5-f882-4782-a116-577a73b931de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:19.737917 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 350/697] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:04:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:04:19.739410 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151afb94-b5f8-4a39-8de1-796921190d9d None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:19.739711 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151afb94-b5f8-4a39-8de1-796921190d9d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:19.739998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151afb94-b5f8-4a39-8de1-796921190d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:19.744200 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5aca5-f882-4782-a116-577a73b931de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:19.745888 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5aca5-f882-4782-a116-577a73b931de None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:19.759674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-151afb94-b5f8-4a39-8de1-796921190d9d None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:19.760973 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-151afb94-b5f8-4a39-8de1-796921190d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:19.762723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-c6f5aca5-f882-4782-a116-577a73b931de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:19.762723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:19.762723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:19.762723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:19.762723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:19.762723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:19.762723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:19.762723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:19.762723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:19.762723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:19.762723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:19.762723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:19.762723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:19.762723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:19.762723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:19.763765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:19.763765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:19.763765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:19.763765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:19.763765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:19.763765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:19.763765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:19.763765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:19.763765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:19.763765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:19.763765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:19.763765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:19.763765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:19.763765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:19.763765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:19.763765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:19.763765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:19.764744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:19.764744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:19.764744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:19.764744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:19.764744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:19.764744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:19.764744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:19.764744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:19.769091 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 348/698] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:19.770012 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5aca5-f882-4782-a116-577a73b931de None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:19.787218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:19.818489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:19.843042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 351/699] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:04:19.844215 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151afb94-b5f8-4a39-8de1-796921190d9d None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:19.875690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5aca5-f882-4782-a116-577a73b931de None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:19.911860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ba5cf-c18f-468b-9e40-9b564934b699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:19.913109 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ba5cf-c18f-468b-9e40-9b564934b699 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:19.913457 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ba5cf-c18f-468b-9e40-9b564934b699 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:19.914040 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ba5cf-c18f-468b-9e40-9b564934b699 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:19.927386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ba5cf-c18f-468b-9e40-9b564934b699 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:19.929448 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ba5cf-c18f-468b-9e40-9b564934b699 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:19.952231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-be4ba5cf-c18f-468b-9e40-9b564934b699 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:19.952231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:19.952231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:19.952231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:19.952231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:19.952231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:19.952231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:19.952231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:19.952231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:19.952231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:19.952231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:19.952231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:19.952231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:19.952231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:19.952231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:19.953401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:19.953401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:19.953401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:19.953401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:19.953401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:19.953401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:19.953401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:19.953401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:19.953401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:19.953401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:19.953401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:19.953401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:19.953401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:19.953401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:19.953401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:19.953401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:19.953401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:19.954513 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:19.954513 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:19.954513 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:19.954513 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:19.954513 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 349/700] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:19.954826 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ba5cf-c18f-468b-9e40-9b564934b699 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:19.978344 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151afb94-b5f8-4a39-8de1-796921190d9d None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:20.017189 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34eaf3-2abe-475d-a4db-b6934ea5f574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:20.018173 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34eaf3-2abe-475d-a4db-b6934ea5f574 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:20.018644 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34eaf3-2abe-475d-a4db-b6934ea5f574 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:20.019108 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34eaf3-2abe-475d-a4db-b6934ea5f574 None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:20.020330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34eaf3-2abe-475d-a4db-b6934ea5f574 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:20.020859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34eaf3-2abe-475d-a4db-b6934ea5f574 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:20.022507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34eaf3-2abe-475d-a4db-b6934ea5f574 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:20.040064 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 352/701] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:20.041289 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34eaf3-2abe-475d-a4db-b6934ea5f574 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:20.069114 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:20.096262 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:20.104129 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ba5cf-c18f-468b-9e40-9b564934b699 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:20.149228 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077814fc-8914-476d-a169-05d55ba4795b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:20.150299 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077814fc-8914-476d-a169-05d55ba4795b None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:20.150655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34eaf3-2abe-475d-a4db-b6934ea5f574 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:20.150895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077814fc-8914-476d-a169-05d55ba4795b None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:20.151341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077814fc-8914-476d-a169-05d55ba4795b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:20.166465 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077814fc-8914-476d-a169-05d55ba4795b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:20.168358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077814fc-8914-476d-a169-05d55ba4795b None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:20.190980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-077814fc-8914-476d-a169-05d55ba4795b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:20.190980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:20.190980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:20.190980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:20.190980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:20.190980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:20.190980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:20.190980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:20.190980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:20.190980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:20.190980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:20.190980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:20.190980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:20.190980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:20.190980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:20.197358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:20.197358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:20.197358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:20.197358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:20.197358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:20.197358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:20.197358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:20.197358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:20.197358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:20.197358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:20.197358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:20.197358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:20.197358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:20.197358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:20.197358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:20.197358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:20.197358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:20.198325 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:20.198325 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:20.198325 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:20.198325 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:20.198325 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 350/702] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:20.198325 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077814fc-8914-476d-a169-05d55ba4795b None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:20.202790 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a5628-12c6-4f09-b007-aeda75b6637e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:20.203557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a5628-12c6-4f09-b007-aeda75b6637e None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:20.203960 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a5628-12c6-4f09-b007-aeda75b6637e None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:20.204387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a5628-12c6-4f09-b007-aeda75b6637e None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:20.205242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a5628-12c6-4f09-b007-aeda75b6637e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:20.205653 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a5628-12c6-4f09-b007-aeda75b6637e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:20.207171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a5628-12c6-4f09-b007-aeda75b6637e None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:20.225179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 353/703] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:20.225179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a5628-12c6-4f09-b007-aeda75b6637e None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:20.308798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:20.325075 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077814fc-8914-476d-a169-05d55ba4795b None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:20.329174 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a5628-12c6-4f09-b007-aeda75b6637e None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:20.362954 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac98b01-d821-4036-b02c-13a4e4b6ebce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:20.364015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac98b01-d821-4036-b02c-13a4e4b6ebce None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:20.364500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac98b01-d821-4036-b02c-13a4e4b6ebce None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:20.364984 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac98b01-d821-4036-b02c-13a4e4b6ebce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:20.366009 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac98b01-d821-4036-b02c-13a4e4b6ebce None admin] RBAC: Adding query filter params (role.id=edf696ae763147cab68434524d1eff9c, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=5b1331b4f0924bb8b137a22a33c9d130) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:20.366523 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac98b01-d821-4036-b02c-13a4e4b6ebce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:20.368270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac98b01-d821-4036-b02c-13a4e4b6ebce None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:20.370288 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86c673-b5f0-48d9-91ed-82c04bbc66c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:20.371102 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86c673-b5f0-48d9-91ed-82c04bbc66c3 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:20.371404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86c673-b5f0-48d9-91ed-82c04bbc66c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:20.371692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86c673-b5f0-48d9-91ed-82c04bbc66c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:20.372436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86c673-b5f0-48d9-91ed-82c04bbc66c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:20.372740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86c673-b5f0-48d9-91ed-82c04bbc66c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:20.374190 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86c673-b5f0-48d9-91ed-82c04bbc66c3 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:20.382941 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 351/704] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:04:20 2022] GET /identity/v3/role_assignments?user.id=5b1331b4f0924bb8b137a22a33c9d130&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=edf696ae763147cab68434524d1eff9c => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:20.382941 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac98b01-d821-4036-b02c-13a4e4b6ebce None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:20.387447 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 354/705] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:20.388467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e67bce-8a60-450e-ad6f-b70645b20354 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:20.388766 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e67bce-8a60-450e-ad6f-b70645b20354 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:20.389051 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e67bce-8a60-450e-ad6f-b70645b20354 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:20.390679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 355/706] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:20.395983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e67bce-8a60-450e-ad6f-b70645b20354 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:20.409950 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:20.519131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e67bce-8a60-450e-ad6f-b70645b20354 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:20.529095 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac98b01-d821-4036-b02c-13a4e4b6ebce None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:20.560052 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5b365-5fe5-46ce-a11a-4437cb306b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:20.560294 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5b365-5fe5-46ce-a11a-4437cb306b81 None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:20.560389 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5b365-5fe5-46ce-a11a-4437cb306b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:20.560662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5b365-5fe5-46ce-a11a-4437cb306b81 None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:20.562075 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b5b365-5fe5-46ce-a11a-4437cb306b81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:20.563577 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b5b365-5fe5-46ce-a11a-4437cb306b81 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:20.575161 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:20.576385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042cd5b1-3cf1-44cb-9902-cdced867604b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:20.577799 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cd5b1-3cf1-44cb-9902-cdced867604b None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:20.577799 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cd5b1-3cf1-44cb-9902-cdced867604b None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:20.577799 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cd5b1-3cf1-44cb-9902-cdced867604b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:20.585656 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042cd5b1-3cf1-44cb-9902-cdced867604b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:20.588321 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042cd5b1-3cf1-44cb-9902-cdced867604b None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:20.601334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-042cd5b1-3cf1-44cb-9902-cdced867604b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:20.601334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:20.601334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:20.601334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:20.601334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:20.601334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:20.601334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:20.601334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:20.601334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:20.601334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:20.601334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:20.601334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:20.601334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:20.601334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:20.601334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:20.602301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:20.602301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:20.602301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:20.602301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:20.602301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:20.602301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:20.602301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:20.602301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:20.602301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:20.602301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:20.602301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:20.602301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:20.602301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:20.602301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:20.602301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:20.602301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:20.602301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:20.603305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:20.603305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:20.603305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:20.603305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:20.603305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:20.603305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:20.603305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:20.603305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:20.603305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 352/707] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:20.603838 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b24aa-1e91-4988-bfd9-a05d1bebae96 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:20.604060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b24aa-1e91-4988-bfd9-a05d1bebae96 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:20.604550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b24aa-1e91-4988-bfd9-a05d1bebae96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:20.622531 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-423b24aa-1e91-4988-bfd9-a05d1bebae96 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:20.623770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-423b24aa-1e91-4988-bfd9-a05d1bebae96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:20.688059 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 356/708] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jun 23 09:04:20 2022] POST /identity/v3/users => generated 254 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:04:20.695064 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5b365-5fe5-46ce-a11a-4437cb306b81 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:20.705635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 353/709] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:04:20.752942 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b24aa-1e91-4988-bfd9-a05d1bebae96 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:20.755394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:20.868303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5b365-5fe5-46ce-a11a-4437cb306b81 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:20.922452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada6682-b77b-48bf-acdd-3adc978726bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:20.923181 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada6682-b77b-48bf-acdd-3adc978726bc None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:20.923474 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada6682-b77b-48bf-acdd-3adc978726bc None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:20.923755 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada6682-b77b-48bf-acdd-3adc978726bc None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:20.924559 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada6682-b77b-48bf-acdd-3adc978726bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:20.924873 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada6682-b77b-48bf-acdd-3adc978726bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:20.926314 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada6682-b77b-48bf-acdd-3adc978726bc None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:20.931689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b24aa-1e91-4988-bfd9-a05d1bebae96 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:20.949711 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 357/710] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:20.951203 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568724fd-7ab5-406d-b71a-8f72c89595a8 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:20.952687 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568724fd-7ab5-406d-b71a-8f72c89595a8 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:20.952687 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568724fd-7ab5-406d-b71a-8f72c89595a8 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:20.956897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 358/711] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:20.957730 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568724fd-7ab5-406d-b71a-8f72c89595a8 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:20.980889 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa82703-ef3f-432e-ab29-f9fc6cee9c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:20.981718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa82703-ef3f-432e-ab29-f9fc6cee9c00 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:20.982337 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa82703-ef3f-432e-ab29-f9fc6cee9c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:20.982628 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa82703-ef3f-432e-ab29-f9fc6cee9c00 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:20.984070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa82703-ef3f-432e-ab29-f9fc6cee9c00 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:20.985487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa82703-ef3f-432e-ab29-f9fc6cee9c00 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:21.001522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-5aa82703-ef3f-432e-ab29-f9fc6cee9c00 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 09:04:21.001522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:21.001522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:04:21.001522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:04:21.001522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:04:21.001522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:04:21.001522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:04:21.001522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:21.001522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 09:04:21.001522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:21.001522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:21.001522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:21.001522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:21.001522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:21.001522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:21.001522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:21.002626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:21.002626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:21.002626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:21.002626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:21.002626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:21.002626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:21.002626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:21.002626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:04:21.002626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:04:21.002626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:04:21.002626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:04:21.002626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:21.002626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:21.002626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:21.002626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:21.002626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:21.003583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:21.003583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:21.003583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:21.003583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:21.003583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:21.003583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:21.003583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:21.003583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:21.003583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:21.003583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:21.003583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:21.003583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:04:21.003583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:21.003583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 09:04:21.003583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:21.003583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 354/712] 10.222.0.52 () {56 vars in 1129 bytes} [Thu Jun 23 09:04:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:21.004484 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42b8be-62cd-4193-83f1-a10dcbd9dbd4 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:21.004484 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42b8be-62cd-4193-83f1-a10dcbd9dbd4 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:21.004623 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42b8be-62cd-4193-83f1-a10dcbd9dbd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:21.024885 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-ef42b8be-62cd-4193-83f1-a10dcbd9dbd4 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:21.026729 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ef42b8be-62cd-4193-83f1-a10dcbd9dbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:21.036913 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:21.089903 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568724fd-7ab5-406d-b71a-8f72c89595a8 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:21.105163 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:21.119018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 355/713] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:04:21.119967 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef42b8be-62cd-4193-83f1-a10dcbd9dbd4 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:21.132491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d1058-c371-4010-ac67-29fe905675fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:21.134272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d1058-c371-4010-ac67-29fe905675fa None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:21.134272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d1058-c371-4010-ac67-29fe905675fa None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:21.134272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d1058-c371-4010-ac67-29fe905675fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:21.134722 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d1058-c371-4010-ac67-29fe905675fa None admin] RBAC: Adding query filter params (role.id=9cd080b50c6f4ee8a54bf9ef1655bd83, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=ff025cb706d243778468ae4570374690) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:21.134962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d1058-c371-4010-ac67-29fe905675fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:21.136400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d1058-c371-4010-ac67-29fe905675fa None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:21.144450 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 359/714] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:04:20 2022] GET /identity/v3/role_assignments?user.id=ff025cb706d243778468ae4570374690&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:21.162111 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d1058-c371-4010-ac67-29fe905675fa None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:21.336712 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef42b8be-62cd-4193-83f1-a10dcbd9dbd4 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:21.366520 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d1058-c371-4010-ac67-29fe905675fa None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:21.404748 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b652f-366e-469e-8526-fc339b4e0fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:21.405350 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b652f-366e-469e-8526-fc339b4e0fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:21.405639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b652f-366e-469e-8526-fc339b4e0fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:21.405942 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b652f-366e-469e-8526-fc339b4e0fe8 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:21.406665 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b652f-366e-469e-8526-fc339b4e0fe8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:21.407272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:21.407461 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b652f-366e-469e-8526-fc339b4e0fe8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:21.408847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b652f-366e-469e-8526-fc339b4e0fe8 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:21.423280 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 356/715] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jun 23 09:04:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:21.430788 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b652f-366e-469e-8526-fc339b4e0fe8 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:21.450889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9511bb0-8691-4d3f-9885-1c64e0462965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:21.451717 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9511bb0-8691-4d3f-9885-1c64e0462965 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:21.452000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9511bb0-8691-4d3f-9885-1c64e0462965 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:21.452277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9511bb0-8691-4d3f-9885-1c64e0462965 None admin] PATH_INFO: `/v3/services/network` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:21.452936 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9511bb0-8691-4d3f-9885-1c64e0462965 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:21.454303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9511bb0-8691-4d3f-9885-1c64e0462965 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:21.475422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-d9511bb0-8691-4d3f-9885-1c64e0462965 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 09:04:21.475422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:21.475422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:04:21.475422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:04:21.475422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:04:21.475422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:04:21.475422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:04:21.475422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:21.475422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 09:04:21.475422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:21.475422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:21.475422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:21.475422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:21.475422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:21.475422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:21.475422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:21.475422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:21.476540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:21.476540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:21.476540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:21.476540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:21.476540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:21.476540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:21.476540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:04:21.476540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:04:21.476540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:04:21.476540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:04:21.476540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:21.476540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:21.476540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:21.476540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:21.476540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:21.476540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:21.477435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:21.477435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:21.477435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:21.477435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:21.477435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:21.477435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:21.477435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:21.477435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:21.477435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:21.477435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:21.477435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:04:21.477435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:21.477435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 09:04:21.477435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:21.489338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 360/716] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:21 2022] GET /identity/v3/services/network => generated 88 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:21.490260 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9511bb0-8691-4d3f-9885-1c64e0462965 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:21.558141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:21.632637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b652f-366e-469e-8526-fc339b4e0fe8 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:21.675399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d570c-b558-479c-a9c4-d997c16264b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:21.675993 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d570c-b558-479c-a9c4-d997c16264b2 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:21.676275 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d570c-b558-479c-a9c4-d997c16264b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:21.676550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d570c-b558-479c-a9c4-d997c16264b2 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:21.677264 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d570c-b558-479c-a9c4-d997c16264b2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:21.677570 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d570c-b558-479c-a9c4-d997c16264b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:21.678956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d570c-b558-479c-a9c4-d997c16264b2 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:21.683911 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9511bb0-8691-4d3f-9885-1c64e0462965 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:21.696289 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 357/717] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jun 23 09:04:21 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:21.734628 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc57a9f0-b9d6-403b-82c5-b5f4ac85cd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:21.735367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57a9f0-b9d6-403b-82c5-b5f4ac85cd98 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:21.735655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57a9f0-b9d6-403b-82c5-b5f4ac85cd98 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:21.735929 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57a9f0-b9d6-403b-82c5-b5f4ac85cd98 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:21.736655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc57a9f0-b9d6-403b-82c5-b5f4ac85cd98 None admin] RBAC: Adding query filter params (name=network) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:21.736967 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc57a9f0-b9d6-403b-82c5-b5f4ac85cd98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:21.738350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc57a9f0-b9d6-403b-82c5-b5f4ac85cd98 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:21.748381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 361/718] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jun 23 09:04:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:21.753809 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d570c-b558-479c-a9c4-d997c16264b2 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:21.871894 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d570c-b558-479c-a9c4-d997c16264b2 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:21.896776 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:21.917665 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63588c8f-a53b-4a81-8980-f89e9b08109d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:21.918886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63588c8f-a53b-4a81-8980-f89e9b08109d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:21.919207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63588c8f-a53b-4a81-8980-f89e9b08109d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:21.919482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63588c8f-a53b-4a81-8980-f89e9b08109d None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:21.920209 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63588c8f-a53b-4a81-8980-f89e9b08109d None admin] RBAC: Adding query filter params (type=network) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:21.920519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63588c8f-a53b-4a81-8980-f89e9b08109d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:21.921949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63588c8f-a53b-4a81-8980-f89e9b08109d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:21.933586 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 358/719] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jun 23 09:04:21 2022] GET /identity/v3/services?type=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:22.910720 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357b705-d658-4a2a-8c05-270166950275 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:22.911274 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357b705-d658-4a2a-8c05-270166950275 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:22.911389 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357b705-d658-4a2a-8c05-270166950275 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:22.912948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 362/720] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:22.918089 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86e104-0542-4d0a-a9d5-8d4677583c14 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:22.918491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86e104-0542-4d0a-a9d5-8d4677583c14 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:22.918671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86e104-0542-4d0a-a9d5-8d4677583c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:22.939930 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-cd86e104-0542-4d0a-a9d5-8d4677583c14 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:22.941120 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-cd86e104-0542-4d0a-a9d5-8d4677583c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:23.025669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 359/721] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:04:23.078029 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357b705-d658-4a2a-8c05-270166950275 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:23.137756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:23.185345 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb87ac-a436-43c7-b936-aa308754bf15 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:23.185662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb87ac-a436-43c7-b936-aa308754bf15 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:23.185837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb87ac-a436-43c7-b936-aa308754bf15 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:23.187256 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 360/722] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:23.198204 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18a749-1f42-43e2-acc6-aa7a9e10ba1e None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:23.198495 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18a749-1f42-43e2-acc6-aa7a9e10ba1e None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:23.198772 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18a749-1f42-43e2-acc6-aa7a9e10ba1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:23.222548 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-ed18a749-1f42-43e2-acc6-aa7a9e10ba1e None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:23.223800 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ed18a749-1f42-43e2-acc6-aa7a9e10ba1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:23.226811 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357b705-d658-4a2a-8c05-270166950275 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:23.293998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76fbfc-6be0-482d-a000-c30e4d717a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:23.295203 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76fbfc-6be0-482d-a000-c30e4d717a67 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:23.295699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76fbfc-6be0-482d-a000-c30e4d717a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:23.296180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76fbfc-6be0-482d-a000-c30e4d717a67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:23.314390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76fbfc-6be0-482d-a000-c30e4d717a67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:23.323590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-ef76fbfc-6be0-482d-a000-c30e4d717a67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:04:23.323590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:23.323590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:23.323590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:23.323590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:23.323590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:23.323590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:23.323590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:23.323590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:23.323590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:23.323590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:23.323590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:23.323590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:23.323590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:23.323590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:23.323590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:23.325131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:23.325131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:23.325131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:23.325131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:23.325131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:23.325131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:23.325131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:23.325131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:23.325131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:23.325131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:23.325131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:23.325131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:23.325131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:23.325131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:23.325131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:23.325131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:23.326087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:23.326087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:23.326087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:23.326087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:23.326087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:23.326087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:23.326087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:23.326087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:23.326087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:23.326087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:23.326087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:04:23.326087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:23.328925 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 363/723] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:04:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:23.331908 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 361/724] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:04:23.348013 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76fbfc-6be0-482d-a000-c30e4d717a67 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:23.361874 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18a749-1f42-43e2-acc6-aa7a9e10ba1e None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:23.446387 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:23.510133 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18a749-1f42-43e2-acc6-aa7a9e10ba1e None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:23.530657 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76fbfc-6be0-482d-a000-c30e4d717a67 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:23.543672 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:23.548459 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d42c62-1be8-45be-bc4d-711aaf7d8af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:23.549406 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d42c62-1be8-45be-bc4d-711aaf7d8af8 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:23.549673 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d42c62-1be8-45be-bc4d-711aaf7d8af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:23.550024 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d42c62-1be8-45be-bc4d-711aaf7d8af8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:23.563814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d42c62-1be8-45be-bc4d-711aaf7d8af8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:23.572604 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff7552-d316-4cb5-b5e0-d7b263909133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:23.573522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff7552-d316-4cb5-b5e0-d7b263909133 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:23.573922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-38d42c62-1be8-45be-bc4d-711aaf7d8af8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:23.573922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:23.573922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:23.573922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:23.573922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:23.573922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:23.573922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:23.573922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:23.573922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:23.573922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:23.573922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:23.573922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:23.573922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:23.573922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:23.573922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:23.573922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:23.575474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:23.575474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:23.575474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:23.575474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:23.575474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:23.575474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:23.575474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:23.575474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:23.575474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:23.575474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:23.575474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:23.575474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:23.575474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:23.575474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:23.575474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:23.575474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:23.576482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:23.576482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:23.576482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:23.576482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:23.576482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:23.576482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:23.576482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:23.576482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:23.576482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:23.576482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:23.576482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:23.576482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:23.577794 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff7552-d316-4cb5-b5e0-d7b263909133 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:23.578322 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff7552-d316-4cb5-b5e0-d7b263909133 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:23.579025 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 362/725] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:04:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:23.579544 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff7552-d316-4cb5-b5e0-d7b263909133 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:23.580054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff7552-d316-4cb5-b5e0-d7b263909133 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:23.580331 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8c971-0c37-4adb-aefe-bc084214b4cd None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:23.580625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8c971-0c37-4adb-aefe-bc084214b4cd None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:23.580912 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8c971-0c37-4adb-aefe-bc084214b4cd None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:23.582332 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff7552-d316-4cb5-b5e0-d7b263909133 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:23.585121 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 363/726] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:23.589422 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8c971-0c37-4adb-aefe-bc084214b4cd None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:23.604855 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 364/727] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:04:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:23.606742 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1afc82-65e0-4b17-b619-b6a5e67a29f0 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:23.607067 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1afc82-65e0-4b17-b619-b6a5e67a29f0 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:23.607383 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1afc82-65e0-4b17-b619-b6a5e67a29f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:23.624484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-8d1afc82-65e0-4b17-b619-b6a5e67a29f0 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:23.625686 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-8d1afc82-65e0-4b17-b619-b6a5e67a29f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:23.703013 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 365/728] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:04:23.703013 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1afc82-65e0-4b17-b619-b6a5e67a29f0 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:23.707876 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8c971-0c37-4adb-aefe-bc084214b4cd None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:23.726263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:23.755832 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcda28-11ef-483f-9cd3-5c3fb3ca4183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:23.756973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcda28-11ef-483f-9cd3-5c3fb3ca4183 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:23.757531 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcda28-11ef-483f-9cd3-5c3fb3ca4183 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:23.758065 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcda28-11ef-483f-9cd3-5c3fb3ca4183 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:23.759051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abcda28-11ef-483f-9cd3-5c3fb3ca4183 None admin] RBAC: Adding query filter params (name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:23.759582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abcda28-11ef-483f-9cd3-5c3fb3ca4183 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:23.761255 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abcda28-11ef-483f-9cd3-5c3fb3ca4183 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:23.771923 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 364/729] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jun 23 09:04:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:23.773038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcda28-11ef-483f-9cd3-5c3fb3ca4183 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:23.820192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:23.825541 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1afc82-65e0-4b17-b619-b6a5e67a29f0 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:23.870227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab11e2-211c-4ef0-a5a3-553ce15cca20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:23.871978 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab11e2-211c-4ef0-a5a3-553ce15cca20 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:23.872479 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab11e2-211c-4ef0-a5a3-553ce15cca20 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:23.872982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab11e2-211c-4ef0-a5a3-553ce15cca20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:23.891098 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab11e2-211c-4ef0-a5a3-553ce15cca20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:23.892912 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab11e2-211c-4ef0-a5a3-553ce15cca20 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:23.913078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-94ab11e2-211c-4ef0-a5a3-553ce15cca20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:23.913078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:23.913078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:23.913078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:23.913078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:23.913078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:23.913078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:23.913078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:23.913078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:23.913078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:23.913078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:23.913078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:23.913078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:23.913078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:23.913078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:23.914522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:23.914522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:23.914522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:23.914522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:23.914522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:23.914522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:23.914522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:23.914522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:23.914522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:23.914522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:23.914522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:23.914522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:23.914522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:23.914522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:23.914522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:23.914522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:23.914522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:23.915948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:23.915948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:23.915948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:23.915948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:23.916185 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcda28-11ef-483f-9cd3-5c3fb3ca4183 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:23.918091 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 366/730] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:23.919219 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab11e2-211c-4ef0-a5a3-553ce15cca20 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:23.965692 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef61f8-73fc-4a97-a35b-df544dfa928f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:23.966422 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef61f8-73fc-4a97-a35b-df544dfa928f None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:23.966712 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef61f8-73fc-4a97-a35b-df544dfa928f None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:23.967010 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef61f8-73fc-4a97-a35b-df544dfa928f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:23.980927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef61f8-73fc-4a97-a35b-df544dfa928f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:23.990837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-43ef61f8-73fc-4a97-a35b-df544dfa928f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:04:23.990837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:23.990837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:23.990837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:23.990837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:23.990837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:23.990837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:23.990837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:23.990837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:23.990837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:23.990837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:23.990837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:23.990837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:23.990837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:23.990837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:23.990837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:23.991886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:23.991886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:23.991886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:23.991886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:23.991886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:23.991886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:23.991886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:23.991886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:23.991886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:23.991886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:23.991886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:23.991886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:23.991886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:23.991886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:23.991886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:23.991886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:23.992819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:23.992819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:23.992819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:23.992819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:23.992819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:23.992819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:23.992819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:23.992819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:23.992819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:23.992819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:23.992819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:04:23.992819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:23.994986 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 365/731] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:04:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:23.996090 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef61f8-73fc-4a97-a35b-df544dfa928f None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:24.031654 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab11e2-211c-4ef0-a5a3-553ce15cca20 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:24.043792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:24.070108 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07871334-9cae-48ca-a127-2dff44f51485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:24.071286 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07871334-9cae-48ca-a127-2dff44f51485 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:24.071783 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07871334-9cae-48ca-a127-2dff44f51485 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:24.072317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07871334-9cae-48ca-a127-2dff44f51485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:24.084514 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07871334-9cae-48ca-a127-2dff44f51485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:24.086234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07871334-9cae-48ca-a127-2dff44f51485 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:24.104805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-07871334-9cae-48ca-a127-2dff44f51485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:24.104805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:24.104805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:24.104805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:24.104805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:24.104805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:24.104805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:24.104805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:24.104805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:24.104805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:24.104805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:24.104805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:24.104805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:24.104805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:24.104805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:24.106349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:24.106349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:24.106349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:24.106349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:24.106349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:24.106349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:24.106349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:24.106349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:24.106349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:24.106349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:24.106349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:24.106349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:24.106349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:24.106349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:24.106349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:24.106349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:24.106349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:24.107343 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:24.107343 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:24.107343 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:24.107343 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:24.109669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 367/732] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:24.110696 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07871334-9cae-48ca-a127-2dff44f51485 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:24.114653 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:24.134721 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef61f8-73fc-4a97-a35b-df544dfa928f None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:24.181708 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132060e6-7671-49a5-9bcb-b278e8b32d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:24.181708 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132060e6-7671-49a5-9bcb-b278e8b32d8f None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:24.181708 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132060e6-7671-49a5-9bcb-b278e8b32d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:24.181708 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132060e6-7671-49a5-9bcb-b278e8b32d8f None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:24.182184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132060e6-7671-49a5-9bcb-b278e8b32d8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:24.182295 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132060e6-7671-49a5-9bcb-b278e8b32d8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:24.183828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132060e6-7671-49a5-9bcb-b278e8b32d8f None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:24.197175 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 366/733] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:24.198156 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132060e6-7671-49a5-9bcb-b278e8b32d8f None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:24.218734 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07871334-9cae-48ca-a127-2dff44f51485 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:24.253819 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328d7d4-6e35-4fc3-8120-9a4073f30566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:24.254495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328d7d4-6e35-4fc3-8120-9a4073f30566 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:24.254787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328d7d4-6e35-4fc3-8120-9a4073f30566 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:24.255117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328d7d4-6e35-4fc3-8120-9a4073f30566 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:24.255857 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4328d7d4-6e35-4fc3-8120-9a4073f30566 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:24.256171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4328d7d4-6e35-4fc3-8120-9a4073f30566 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:24.257535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4328d7d4-6e35-4fc3-8120-9a4073f30566 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:24.265565 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 368/734] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:04:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:24.266784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2347692-fffc-477b-b91e-e1dce05f6976 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:24.267104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2347692-fffc-477b-b91e-e1dce05f6976 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:24.267385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2347692-fffc-477b-b91e-e1dce05f6976 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:24.268763 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 369/735] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:24.269300 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2347692-fffc-477b-b91e-e1dce05f6976 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:24.307905 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:24.317364 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132060e6-7671-49a5-9bcb-b278e8b32d8f None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:24.350929 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:24.359436 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287a6cf-d849-4721-a55b-0ce1ee71df47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:24.360123 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287a6cf-d849-4721-a55b-0ce1ee71df47 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:24.360410 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287a6cf-d849-4721-a55b-0ce1ee71df47 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:24.360694 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287a6cf-d849-4721-a55b-0ce1ee71df47 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:24.361596 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7287a6cf-d849-4721-a55b-0ce1ee71df47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:24.361752 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7287a6cf-d849-4721-a55b-0ce1ee71df47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:24.363230 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7287a6cf-d849-4721-a55b-0ce1ee71df47 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:24.380543 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 367/736] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:24.381511 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287a6cf-d849-4721-a55b-0ce1ee71df47 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:24.389197 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2347692-fffc-477b-b91e-e1dce05f6976 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:24.428545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d447f8-05a7-48dc-a9b7-7ba7fb267d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:24.429711 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d447f8-05a7-48dc-a9b7-7ba7fb267d36 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:24.430001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d447f8-05a7-48dc-a9b7-7ba7fb267d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:24.430605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d447f8-05a7-48dc-a9b7-7ba7fb267d36 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:24.438989 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d447f8-05a7-48dc-a9b7-7ba7fb267d36 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:24.440441 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d447f8-05a7-48dc-a9b7-7ba7fb267d36 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:24.454823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-d8d447f8-05a7-48dc-a9b7-7ba7fb267d36 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 09:04:24.454823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:24.454823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:24.454823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:24.454823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:24.454823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:24.454823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:24.454823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:24.454823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:24.454823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:24.454823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:24.454823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:24.454823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:24.454823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:24.454823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:24.454823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:24.455968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:24.455968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:24.455968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:24.455968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:24.455968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:24.455968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:24.455968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:24.455968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:24.455968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:24.455968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:24.455968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:24.455968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:24.455968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:24.455968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:24.455968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:24.455968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:24.455968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:24.455968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:24.456980 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:24.456980 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:24.456980 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:24.456980 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:24.456980 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:24.456980 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:24.456980 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:24.456980 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:24.456980 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 09:04:24.456980 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:24.461327 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 370/737] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:04:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:24.462216 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c805c-c224-4cb0-b859-e889c2247fce None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:24.462508 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c805c-c224-4cb0-b859-e889c2247fce None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:24.462788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c805c-c224-4cb0-b859-e889c2247fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:24.479568 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-b85c805c-c224-4cb0-b859-e889c2247fce None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:24.481238 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b85c805c-c224-4cb0-b859-e889c2247fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:24.491680 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287a6cf-d849-4721-a55b-0ce1ee71df47 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:24.524202 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7eb65-57bb-49ca-9e64-122495d464fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:24.524993 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7eb65-57bb-49ca-9e64-122495d464fd None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:24.525283 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7eb65-57bb-49ca-9e64-122495d464fd None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:24.525564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7eb65-57bb-49ca-9e64-122495d464fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:24.536431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7eb65-57bb-49ca-9e64-122495d464fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:24.537883 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7eb65-57bb-49ca-9e64-122495d464fd None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:24.546238 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 371/738] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:04:24.547092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c805c-c224-4cb0-b859-e889c2247fce None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:24.556183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-acb7eb65-57bb-49ca-9e64-122495d464fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:24.556183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:24.556183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:24.556183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:24.556183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:24.556183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:24.556183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:24.556183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:24.556183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:24.556183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:24.556183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:24.556183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:24.556183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:24.556183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:24.556183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:24.557152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:24.557152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:24.557152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:24.557152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:24.557152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:24.557152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:24.557152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:24.557152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:24.557152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:24.557152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:24.557152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:24.557152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:24.557152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:24.557152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:24.557152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:24.557152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:24.557152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:24.558086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:24.558086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:24.558086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:24.558086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:24.558086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 368/739] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:24.558388 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7eb65-57bb-49ca-9e64-122495d464fd None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:24.563593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:24.609565 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:24.653235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c805c-c224-4cb0-b859-e889c2247fce None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:24.660384 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7eb65-57bb-49ca-9e64-122495d464fd None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:24.686009 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff517c-0412-4acb-80e5-5971b29df00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:24.686830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff517c-0412-4acb-80e5-5971b29df00c None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:24.687158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff517c-0412-4acb-80e5-5971b29df00c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:24.687438 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff517c-0412-4acb-80e5-5971b29df00c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:24.693743 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9996b3-c11b-4caf-a294-5a4ae4131a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:24.694438 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9996b3-c11b-4caf-a294-5a4ae4131a4f None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:24.694731 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9996b3-c11b-4caf-a294-5a4ae4131a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:24.694823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff517c-0412-4acb-80e5-5971b29df00c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:24.695048 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9996b3-c11b-4caf-a294-5a4ae4131a4f None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:24.695911 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9996b3-c11b-4caf-a294-5a4ae4131a4f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:24.696231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9996b3-c11b-4caf-a294-5a4ae4131a4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:24.696331 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff517c-0412-4acb-80e5-5971b29df00c None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:24.697697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9996b3-c11b-4caf-a294-5a4ae4131a4f None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:24.712753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-95ff517c-0412-4acb-80e5-5971b29df00c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 09:04:24.712753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:24.712753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:24.712753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:24.712753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:24.712753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:24.712753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:24.712753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:24.712753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:24.712753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:24.712753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:24.712753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:24.712753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:24.712753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:24.712753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:24.712753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:24.713956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:24.713956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:24.713956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:24.713956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:24.713956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:24.713956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:24.713956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:24.713956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:24.713956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:24.713956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:24.713956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:24.713956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:24.713956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:24.713956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:24.713956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:24.713956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:24.713956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:24.713956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:24.715027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:24.715027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:24.715027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:24.715027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:24.715027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:24.715027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:24.715027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:24.715027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:24.715027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 09:04:24.715027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:24.715027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 372/740] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jun 23 09:04:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:24.715657 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fb699-9759-466b-ae78-8b4e3325f298 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:24.715931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fb699-9759-466b-ae78-8b4e3325f298 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:24.716217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fb699-9759-466b-ae78-8b4e3325f298 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:24.717390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 373/741] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:24.718084 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fb699-9759-466b-ae78-8b4e3325f298 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:24.722432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 369/742] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jun 23 09:04:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:24.722432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9996b3-c11b-4caf-a294-5a4ae4131a4f None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:24.779489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:24.819202 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9996b3-c11b-4caf-a294-5a4ae4131a4f None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:24.819406 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fb699-9759-466b-ae78-8b4e3325f298 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:24.835611 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:24.851944 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21231c58-8d09-4d69-b035-cf5bde56ca08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:24.852320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f8de60-de1b-4508-8677-93d633d9a45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:24.852602 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21231c58-8d09-4d69-b035-cf5bde56ca08 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:24.852888 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21231c58-8d09-4d69-b035-cf5bde56ca08 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:24.852984 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8de60-de1b-4508-8677-93d633d9a45f None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:24.853163 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21231c58-8d09-4d69-b035-cf5bde56ca08 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:24.853256 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8de60-de1b-4508-8677-93d633d9a45f None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:24.853513 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8de60-de1b-4508-8677-93d633d9a45f None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:24.853881 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21231c58-8d09-4d69-b035-cf5bde56ca08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:24.854184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21231c58-8d09-4d69-b035-cf5bde56ca08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:24.854276 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f8de60-de1b-4508-8677-93d633d9a45f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:24.856144 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21231c58-8d09-4d69-b035-cf5bde56ca08 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:24.856255 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f8de60-de1b-4508-8677-93d633d9a45f None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:24.867570 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 374/743] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:24.868348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21231c58-8d09-4d69-b035-cf5bde56ca08 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:24.941320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 370/744] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jun 23 09:04:24 2022] POST /identity/v3/services => generated 251 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:04:24.942597 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394725e-0b06-40b1-b80d-4f43e7e03a54 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:24.942892 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394725e-0b06-40b1-b80d-4f43e7e03a54 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:24.943206 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394725e-0b06-40b1-b80d-4f43e7e03a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:24.960780 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-f394725e-0b06-40b1-b80d-4f43e7e03a54 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:24.962028 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-f394725e-0b06-40b1-b80d-4f43e7e03a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:24.981881 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21231c58-8d09-4d69-b035-cf5bde56ca08 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:25.036442 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69857e-a1be-46a3-a87c-616d8ca0bcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:25.037051 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69857e-a1be-46a3-a87c-616d8ca0bcd9 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:25.037339 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69857e-a1be-46a3-a87c-616d8ca0bcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:25.037620 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69857e-a1be-46a3-a87c-616d8ca0bcd9 None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:25.038477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f69857e-a1be-46a3-a87c-616d8ca0bcd9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:25.038803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f69857e-a1be-46a3-a87c-616d8ca0bcd9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:25.040302 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f69857e-a1be-46a3-a87c-616d8ca0bcd9 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:25.051160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 371/745] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:24 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 09:04:25.051160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394725e-0b06-40b1-b80d-4f43e7e03a54 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:25.076428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 375/746] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jun 23 09:04:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:25.076428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69857e-a1be-46a3-a87c-616d8ca0bcd9 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:25.108096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:25.109745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:25.167324 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394725e-0b06-40b1-b80d-4f43e7e03a54 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:25.187843 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69857e-a1be-46a3-a87c-616d8ca0bcd9 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:25.204844 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9b53e-f739-4f6f-b71f-c673b1ae92e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:25.205500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9b53e-f739-4f6f-b71f-c673b1ae92e9 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:25.205781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9b53e-f739-4f6f-b71f-c673b1ae92e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:25.206112 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9b53e-f739-4f6f-b71f-c673b1ae92e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:25.225316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b327be5-8a12-4a3b-914c-c06057d4b911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:25.226326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b327be5-8a12-4a3b-914c-c06057d4b911 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:25.227127 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b327be5-8a12-4a3b-914c-c06057d4b911 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:25.227127 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b327be5-8a12-4a3b-914c-c06057d4b911 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:25.235330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b327be5-8a12-4a3b-914c-c06057d4b911 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:25.236776 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b327be5-8a12-4a3b-914c-c06057d4b911 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:25.251234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-1b327be5-8a12-4a3b-914c-c06057d4b911 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 09:04:25.251234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:25.251234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:25.251234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:25.251234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:25.251234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:25.251234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:25.251234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:25.251234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:25.251234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:25.251234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:25.251234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:25.251234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:25.251234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:25.251234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:25.251234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:25.252531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:25.252531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:25.252531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:25.252531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:25.252531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:25.252531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:25.252531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:25.252531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:25.252531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:25.252531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:25.252531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:25.252531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:25.252531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:25.252531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:25.252531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:25.252531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:25.252531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:25.252531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:25.253770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:25.253770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:25.253770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:25.253770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:25.253770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:25.253770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:25.253770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:25.253770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:25.253770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 09:04:25.253770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:25.258703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 376/747] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jun 23 09:04:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:25.259677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b327be5-8a12-4a3b-914c-c06057d4b911 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:25.284867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9b53e-f739-4f6f-b71f-c673b1ae92e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:25.286524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9b53e-f739-4f6f-b71f-c673b1ae92e9 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:25.339383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:25.361282 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:25.373242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b327be5-8a12-4a3b-914c-c06057d4b911 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:25.397949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 372/748] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:04:25 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 09:04:25.397949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9b53e-f739-4f6f-b71f-c673b1ae92e9 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:25.417629 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08058c-babb-440c-a421-a03aaa13ac46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:25.418636 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08058c-babb-440c-a421-a03aaa13ac46 None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:25.418924 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08058c-babb-440c-a421-a03aaa13ac46 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:25.419269 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08058c-babb-440c-a421-a03aaa13ac46 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:25.420078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08058c-babb-440c-a421-a03aaa13ac46 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:25.421583 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08058c-babb-440c-a421-a03aaa13ac46 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:25.493560 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9b53e-f739-4f6f-b71f-c673b1ae92e9 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:25.527939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 377/749] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jun 23 09:04:25 2022] POST /identity/v3/services => generated 240 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:04:25.528904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08058c-babb-440c-a421-a03aaa13ac46 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:25.540928 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb7d1a-5876-4326-bc22-1acb057f011c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:25.541950 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb7d1a-5876-4326-bc22-1acb057f011c None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:25.542511 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb7d1a-5876-4326-bc22-1acb057f011c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:25.543117 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb7d1a-5876-4326-bc22-1acb057f011c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:25.583355 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:25.623182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb7d1a-5876-4326-bc22-1acb057f011c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:25.625069 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb7d1a-5876-4326-bc22-1acb057f011c None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:25.667253 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08058c-babb-440c-a421-a03aaa13ac46 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:25.718615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7aa2b6-04a2-4d62-b068-5c8f759289d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:25.719579 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7aa2b6-04a2-4d62-b068-5c8f759289d1 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:25.720018 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7aa2b6-04a2-4d62-b068-5c8f759289d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:25.720458 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7aa2b6-04a2-4d62-b068-5c8f759289d1 None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:25.721308 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7aa2b6-04a2-4d62-b068-5c8f759289d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:25.721623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7aa2b6-04a2-4d62-b068-5c8f759289d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:25.723157 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7aa2b6-04a2-4d62-b068-5c8f759289d1 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:25.741258 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 378/750] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jun 23 09:04:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:25.742105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7aa2b6-04a2-4d62-b068-5c8f759289d1 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:25.754135 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 373/751] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:04:25 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 09:04:25.756081 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb7d1a-5876-4326-bc22-1acb057f011c None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:25.757569 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:25.814304 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:25.862147 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7aa2b6-04a2-4d62-b068-5c8f759289d1 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:25.878688 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb7d1a-5876-4326-bc22-1acb057f011c None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:25.902950 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cffb1-6d13-45e1-aadf-6592a33a169d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:25.903735 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cffb1-6d13-45e1-aadf-6592a33a169d None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:25.904097 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cffb1-6d13-45e1-aadf-6592a33a169d None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:25.904447 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cffb1-6d13-45e1-aadf-6592a33a169d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:25.914994 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804cffb1-6d13-45e1-aadf-6592a33a169d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:25.915659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872c2ee-203e-4906-80ac-b668baf382fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:25.916461 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872c2ee-203e-4906-80ac-b668baf382fa None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:25.916624 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804cffb1-6d13-45e1-aadf-6592a33a169d None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:25.916726 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872c2ee-203e-4906-80ac-b668baf382fa None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:25.917006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872c2ee-203e-4906-80ac-b668baf382fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:25.928206 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5872c2ee-203e-4906-80ac-b668baf382fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:25.929672 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5872c2ee-203e-4906-80ac-b668baf382fa None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:25.935539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-804cffb1-6d13-45e1-aadf-6592a33a169d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:25.935539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:25.935539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:25.935539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:25.935539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:25.935539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:25.935539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:25.935539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:25.935539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:25.935539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:25.935539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:25.935539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:25.935539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:25.935539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:25.935539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:25.936482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:25.936482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:25.936482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:25.936482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:25.936482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:25.936482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:25.936482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:25.936482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:25.936482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:25.936482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:25.936482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:25.936482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:25.936482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:25.936482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:25.936482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:25.936482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:25.936482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:25.937456 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:25.937456 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:25.937456 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:25.937456 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:25.937456 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 379/752] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:25.937771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cffb1-6d13-45e1-aadf-6592a33a169d None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:25.948813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-5872c2ee-203e-4906-80ac-b668baf382fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:25.948813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:25.948813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:25.948813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:25.948813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:25.948813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:25.948813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:25.948813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:25.948813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:25.948813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:25.948813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:25.948813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:25.948813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:25.948813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:25.948813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:25.950940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:25.950940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:25.950940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:25.950940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:25.950940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:25.950940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:25.950940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:25.950940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:25.950940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:25.950940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:25.950940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:25.950940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:25.950940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:25.950940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:25.950940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:25.950940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:25.950940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:25.953172 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:25.953172 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:25.953172 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:25.953172 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:25.953172 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 374/753] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:25.953172 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872c2ee-203e-4906-80ac-b668baf382fa None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:25.999302 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:26.040608 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cffb1-6d13-45e1-aadf-6592a33a169d None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:26.048601 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:26.055372 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872c2ee-203e-4906-80ac-b668baf382fa None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:26.079602 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436e2c8-b4b0-4d69-8aa5-cc6d142171ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:26.080688 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436e2c8-b4b0-4d69-8aa5-cc6d142171ed None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:26.081127 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436e2c8-b4b0-4d69-8aa5-cc6d142171ed None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:26.081535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436e2c8-b4b0-4d69-8aa5-cc6d142171ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:26.091545 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9210915-5338-4640-b386-a5ff5fdd94c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:26.092329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9210915-5338-4640-b386-a5ff5fdd94c0 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:26.092689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9210915-5338-4640-b386-a5ff5fdd94c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:26.093032 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9210915-5338-4640-b386-a5ff5fdd94c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:26.093833 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9210915-5338-4640-b386-a5ff5fdd94c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:26.094207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9210915-5338-4640-b386-a5ff5fdd94c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:26.095551 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436e2c8-b4b0-4d69-8aa5-cc6d142171ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:26.095794 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9210915-5338-4640-b386-a5ff5fdd94c0 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:26.097056 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436e2c8-b4b0-4d69-8aa5-cc6d142171ed None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:26.108501 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 375/754] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:26.112388 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9210915-5338-4640-b386-a5ff5fdd94c0 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:26.123465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-d436e2c8-b4b0-4d69-8aa5-cc6d142171ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:26.123465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:26.123465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:26.123465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:26.123465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:26.123465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:26.123465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:26.123465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:26.123465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:26.123465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:26.123465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:26.123465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:26.123465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:26.123465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:26.123465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:26.124679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:26.124679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:26.124679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:26.124679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:26.124679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:26.124679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:26.124679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:26.124679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:26.124679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:26.124679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:26.124679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:26.124679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:26.124679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:26.124679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:26.124679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:26.124679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:26.124679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:26.125643 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:26.125643 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:26.125643 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:26.125643 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:26.128479 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 380/755] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:26.129598 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436e2c8-b4b0-4d69-8aa5-cc6d142171ed None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:26.235444 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9210915-5338-4640-b386-a5ff5fdd94c0 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:26.245843 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436e2c8-b4b0-4d69-8aa5-cc6d142171ed None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:26.258878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:26.272851 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ad04f-b472-4a59-9d16-562985839f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:26.273618 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ad04f-b472-4a59-9d16-562985839f4a None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:26.273974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ad04f-b472-4a59-9d16-562985839f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:26.274401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ad04f-b472-4a59-9d16-562985839f4a None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:26.275272 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ad04f-b472-4a59-9d16-562985839f4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:26.275577 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ad04f-b472-4a59-9d16-562985839f4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:26.277012 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ad04f-b472-4a59-9d16-562985839f4a None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:26.282808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eed60c-be6a-4e7c-a75b-97b294fb936c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:26.283864 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eed60c-be6a-4e7c-a75b-97b294fb936c None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:26.284336 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eed60c-be6a-4e7c-a75b-97b294fb936c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:26.284789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eed60c-be6a-4e7c-a75b-97b294fb936c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:26.294531 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 376/756] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:26.294531 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ad04f-b472-4a59-9d16-562985839f4a None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:26.297362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eed60c-be6a-4e7c-a75b-97b294fb936c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:26.299179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eed60c-be6a-4e7c-a75b-97b294fb936c None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:26.308985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:26.314242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-a5eed60c-be6a-4e7c-a75b-97b294fb936c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:26.314242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:26.314242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:26.314242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:26.314242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:26.314242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:26.314242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:26.314242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:26.314242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:26.314242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:26.314242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:26.314242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:26.314242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:26.314242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:26.314242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:26.319001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:26.319001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:26.319001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:26.319001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:26.319001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:26.319001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:26.319001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:26.319001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:26.319001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:26.319001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:26.319001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:26.319001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:26.319001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:26.319001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:26.319001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:26.319001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:26.319001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:26.320212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:26.320212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:26.320212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:26.320212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:26.320212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:26.320212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:26.320212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:26.320212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:26.320212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 381/757] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:26.320212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eed60c-be6a-4e7c-a75b-97b294fb936c None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:26.406817 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ad04f-b472-4a59-9d16-562985839f4a None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:26.423195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eed60c-be6a-4e7c-a75b-97b294fb936c None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:26.444067 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bcf0e-8d3a-4d71-bb09-ec948b46f0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:26.444813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bcf0e-8d3a-4d71-bb09-ec948b46f0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:26.445167 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bcf0e-8d3a-4d71-bb09-ec948b46f0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:26.445534 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bcf0e-8d3a-4d71-bb09-ec948b46f0a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:26.446421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bcf0e-8d3a-4d71-bb09-ec948b46f0a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:26.446794 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bcf0e-8d3a-4d71-bb09-ec948b46f0a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:26.448371 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bcf0e-8d3a-4d71-bb09-ec948b46f0a7 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:26.458527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875f54d-af3c-44ec-a8a5-816ad1d68566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:26.459681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875f54d-af3c-44ec-a8a5-816ad1d68566 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:26.460041 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875f54d-af3c-44ec-a8a5-816ad1d68566 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:26.460391 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875f54d-af3c-44ec-a8a5-816ad1d68566 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:26.465989 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 377/758] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:26.466948 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bcf0e-8d3a-4d71-bb09-ec948b46f0a7 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:26.467817 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875f54d-af3c-44ec-a8a5-816ad1d68566 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:26.469338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875f54d-af3c-44ec-a8a5-816ad1d68566 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:26.483745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-a875f54d-af3c-44ec-a8a5-816ad1d68566 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:26.483745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:26.483745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:26.483745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:26.483745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:26.483745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:26.483745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:26.483745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:26.483745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:26.483745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:26.483745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:26.483745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:26.483745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:26.483745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:26.483745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:26.484921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:26.484921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:26.484921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:26.484921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:26.484921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:26.484921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:26.484921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:26.484921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:26.484921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:26.484921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:26.484921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:26.484921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:26.484921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:26.484921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:26.484921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:26.484921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:26.484921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:26.486094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:26.486094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:26.486094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:26.486094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:26.486094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:26.486094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:26.486094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:26.486094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:26.492356 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 382/759] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:26.493180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875f54d-af3c-44ec-a8a5-816ad1d68566 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:26.535289 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:26.545560 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:26.571412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bcf0e-8d3a-4d71-bb09-ec948b46f0a7 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:26.595841 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875f54d-af3c-44ec-a8a5-816ad1d68566 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:26.608237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ede157-2559-4e43-8383-238d7def0316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:26.609551 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ede157-2559-4e43-8383-238d7def0316 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:26.610349 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ede157-2559-4e43-8383-238d7def0316 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:26.610784 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ede157-2559-4e43-8383-238d7def0316 None admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:26.611806 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ede157-2559-4e43-8383-238d7def0316 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:26.612259 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ede157-2559-4e43-8383-238d7def0316 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:26.613861 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ede157-2559-4e43-8383-238d7def0316 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:26.631606 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63869177-62fa-41ba-af5c-d7f0f664e124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:26.632010 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 378/760] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:26.632664 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63869177-62fa-41ba-af5c-d7f0f664e124 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:26.633070 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63869177-62fa-41ba-af5c-d7f0f664e124 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:26.633428 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ede157-2559-4e43-8383-238d7def0316 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:26.633573 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63869177-62fa-41ba-af5c-d7f0f664e124 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:26.640864 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63869177-62fa-41ba-af5c-d7f0f664e124 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:26.642449 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63869177-62fa-41ba-af5c-d7f0f664e124 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:26.658361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-63869177-62fa-41ba-af5c-d7f0f664e124 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:26.658361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:26.658361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:26.658361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:26.658361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:26.658361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:26.658361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:26.658361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:26.658361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:26.658361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:26.658361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:26.658361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:26.658361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:26.658361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:26.658361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:26.659535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:26.659535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:26.659535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:26.659535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:26.659535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:26.659535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:26.659535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:26.659535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:26.659535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:26.659535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:26.659535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:26.659535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:26.659535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:26.659535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:26.659535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:26.659535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:26.659535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:26.660701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:26.660701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:26.660701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:26.660701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:26.660701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:26.660701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:26.660701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:26.660701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:26.666708 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 383/761] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:26.666708 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63869177-62fa-41ba-af5c-d7f0f664e124 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:26.747258 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ede157-2559-4e43-8383-238d7def0316 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:26.763347 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:26.765682 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63869177-62fa-41ba-af5c-d7f0f664e124 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:26.785246 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5cb35-905d-4036-ba4f-049aef7762ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:26.785924 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5cb35-905d-4036-ba4f-049aef7762ad None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:26.786205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5cb35-905d-4036-ba4f-049aef7762ad None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:26.786478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5cb35-905d-4036-ba4f-049aef7762ad None admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:26.787397 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5cb35-905d-4036-ba4f-049aef7762ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:26.787707 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5cb35-905d-4036-ba4f-049aef7762ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:26.789191 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5cb35-905d-4036-ba4f-049aef7762ad None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:26.791072 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:26.807418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbb25e-3378-433a-ad1e-6ba8d00fa514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:26.807418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbb25e-3378-433a-ad1e-6ba8d00fa514 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:26.807418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbb25e-3378-433a-ad1e-6ba8d00fa514 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:26.807418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbb25e-3378-433a-ad1e-6ba8d00fa514 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:26.809922 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecbb25e-3378-433a-ad1e-6ba8d00fa514 None admin] RBAC: Adding query filter params (role.id=9cd080b50c6f4ee8a54bf9ef1655bd83, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=5b1331b4f0924bb8b137a22a33c9d130) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:26.809922 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecbb25e-3378-433a-ad1e-6ba8d00fa514 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:26.810101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 379/762] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:26.810101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5cb35-905d-4036-ba4f-049aef7762ad None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:26.810289 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecbb25e-3378-433a-ad1e-6ba8d00fa514 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:26.822941 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 384/763] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:04:26 2022] GET /identity/v3/role_assignments?user.id=5b1331b4f0924bb8b137a22a33c9d130&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:26.822941 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbb25e-3378-433a-ad1e-6ba8d00fa514 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:26.925038 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbb25e-3378-433a-ad1e-6ba8d00fa514 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:26.962844 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc199c9-c594-467a-8bb1-0d2b5eb0bf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:26.963968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc199c9-c594-467a-8bb1-0d2b5eb0bf73 None admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:26.964359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc199c9-c594-467a-8bb1-0d2b5eb0bf73 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:26.965240 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc199c9-c594-467a-8bb1-0d2b5eb0bf73 None admin] PATH_INFO: `/v3/projects/93b5040533524222b126bbe93afc56d1/users/ff025cb706d243778468ae4570374690/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:26.967590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc199c9-c594-467a-8bb1-0d2b5eb0bf73 None admin] RBAC: Authorizing `identity:create_grant(project_id=93b5040533524222b126bbe93afc56d1, user_id=ff025cb706d243778468ae4570374690, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:26.969150 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc199c9-c594-467a-8bb1-0d2b5eb0bf73 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:26.989512 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5cb35-905d-4036-ba4f-049aef7762ad None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:27.027620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75529d36-7585-4c5b-8db9-ccee77ec5314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:27.028305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75529d36-7585-4c5b-8db9-ccee77ec5314 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:27.033547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75529d36-7585-4c5b-8db9-ccee77ec5314 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:27.033547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75529d36-7585-4c5b-8db9-ccee77ec5314 None admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:27.033547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75529d36-7585-4c5b-8db9-ccee77ec5314 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:27.033547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75529d36-7585-4c5b-8db9-ccee77ec5314 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:27.033547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75529d36-7585-4c5b-8db9-ccee77ec5314 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:27.046301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 380/764] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:27.058432 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 385/765] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:04:26 2022] PUT /identity/v3/projects/93b5040533524222b126bbe93afc56d1/users/ff025cb706d243778468ae4570374690/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:04:27.059658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75529d36-7585-4c5b-8db9-ccee77ec5314 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:27.067587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50261312-dd4a-4c28-8e06-070e2f7d110c None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:27.067587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50261312-dd4a-4c28-8e06-070e2f7d110c None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:27.067911 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50261312-dd4a-4c28-8e06-070e2f7d110c None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:27.069758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 386/766] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:04:27.075801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af816e90-6855-4b81-aa68-e1efc62190e5 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:27.076232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af816e90-6855-4b81-aa68-e1efc62190e5 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:27.076654 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af816e90-6855-4b81-aa68-e1efc62190e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:27.095111 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-af816e90-6855-4b81-aa68-e1efc62190e5 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:27.096757 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-af816e90-6855-4b81-aa68-e1efc62190e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:27.116685 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:27.140985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:27.214126 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75529d36-7585-4c5b-8db9-ccee77ec5314 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:27.220509 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 387/767] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 09:04:27.253142 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af816e90-6855-4b81-aa68-e1efc62190e5 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:27.270721 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420b428-fcf6-4f39-be53-e01f1b7f1d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:27.271418 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420b428-fcf6-4f39-be53-e01f1b7f1d2a None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:27.271704 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420b428-fcf6-4f39-be53-e01f1b7f1d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:27.271973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420b428-fcf6-4f39-be53-e01f1b7f1d2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:27.272815 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420b428-fcf6-4f39-be53-e01f1b7f1d2a None admin] RBAC: Adding query filter params (role.id=edf696ae763147cab68434524d1eff9c, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=cfbaba84402d4c31bafcd12133d74c03) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:27.273126 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420b428-fcf6-4f39-be53-e01f1b7f1d2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:27.274632 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420b428-fcf6-4f39-be53-e01f1b7f1d2a None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:27.281083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 381/768] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:04:27 2022] GET /identity/v3/role_assignments?user.id=cfbaba84402d4c31bafcd12133d74c03&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=edf696ae763147cab68434524d1eff9c => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:27.372864 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af816e90-6855-4b81-aa68-e1efc62190e5 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:27.412010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a44c86-09ff-40af-a3e1-80fa99ce76e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:27.412860 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a44c86-09ff-40af-a3e1-80fa99ce76e5 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:27.413157 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a44c86-09ff-40af-a3e1-80fa99ce76e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:27.413439 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a44c86-09ff-40af-a3e1-80fa99ce76e5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:27.414123 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a44c86-09ff-40af-a3e1-80fa99ce76e5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:27.415583 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a44c86-09ff-40af-a3e1-80fa99ce76e5 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:27.428948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-36a44c86-09ff-40af-a3e1-80fa99ce76e5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 09:04:27.428948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:27.428948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:04:27.428948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:04:27.428948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:04:27.428948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:04:27.428948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:04:27.428948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:27.428948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 09:04:27.428948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:27.428948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:27.428948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:27.428948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:27.428948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:27.428948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:27.428948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:27.430014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:27.430014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:27.430014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:27.430014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:27.430014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:27.430014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:27.430014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:27.430014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:04:27.430014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:04:27.430014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:04:27.430014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:04:27.430014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:27.430014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:27.430014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:27.430014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:27.430014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:27.430972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:27.430972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:27.430972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:27.430972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:27.430972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:27.430972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:27.430972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:27.430972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:27.430972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:27.430972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:27.430972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:27.430972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:04:27.430972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:27.430972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 09:04:27.430972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:27.430972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 388/769] 10.222.0.52 () {56 vars in 1129 bytes} [Thu Jun 23 09:04:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:27.436692 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420b428-fcf6-4f39-be53-e01f1b7f1d2a None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:27.552826 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420b428-fcf6-4f39-be53-e01f1b7f1d2a None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:27.571180 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:27.588016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9201636-737e-47e9-abfb-8e80e5d8ae01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:27.588696 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9201636-737e-47e9-abfb-8e80e5d8ae01 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:27.588964 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9201636-737e-47e9-abfb-8e80e5d8ae01 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:27.589241 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9201636-737e-47e9-abfb-8e80e5d8ae01 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:27.590472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9201636-737e-47e9-abfb-8e80e5d8ae01 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:27.590780 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9201636-737e-47e9-abfb-8e80e5d8ae01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:27.592186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9201636-737e-47e9-abfb-8e80e5d8ae01 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:27.597321 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 382/770] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jun 23 09:04:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:27.602476 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a44c86-09ff-40af-a3e1-80fa99ce76e5 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:27.636472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:27.759408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a44c86-09ff-40af-a3e1-80fa99ce76e5 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:27.804510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478d706-5b72-4963-889e-5d41d9946a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:27.805179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478d706-5b72-4963-889e-5d41d9946a13 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:27.805482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478d706-5b72-4963-889e-5d41d9946a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:27.805775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478d706-5b72-4963-889e-5d41d9946a13 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:27.806508 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478d706-5b72-4963-889e-5d41d9946a13 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:27.806831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478d706-5b72-4963-889e-5d41d9946a13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:27.816303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478d706-5b72-4963-889e-5d41d9946a13 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:27.823920 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 389/771] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jun 23 09:04:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:27.829793 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9201636-737e-47e9-abfb-8e80e5d8ae01 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:27.917264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62099d-5c80-4bb2-8ecc-061e9e387a47 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:27.917588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62099d-5c80-4bb2-8ecc-061e9e387a47 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:27.918371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62099d-5c80-4bb2-8ecc-061e9e387a47 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:27.926037 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 390/772] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:27.928620 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38768df9-22d7-4dba-9dca-40944364b49b None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:27.928927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38768df9-22d7-4dba-9dca-40944364b49b None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:27.929315 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38768df9-22d7-4dba-9dca-40944364b49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:27.948164 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9201636-737e-47e9-abfb-8e80e5d8ae01 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:27.956159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-38768df9-22d7-4dba-9dca-40944364b49b None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:27.957376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-38768df9-22d7-4dba-9dca-40944364b49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:27.983620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b0521-d322-44cf-a763-17bb535409e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:27.984285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b0521-d322-44cf-a763-17bb535409e6 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:27.984571 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b0521-d322-44cf-a763-17bb535409e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:27.984849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b0521-d322-44cf-a763-17bb535409e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:27.985640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b0521-d322-44cf-a763-17bb535409e6 None admin] RBAC: Adding query filter params (interface=public, service_id=08675e3c7b664b9d87e17d2d7da2269e, region_id=RegionOne) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:27.985953 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b0521-d322-44cf-a763-17bb535409e6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:27.987368 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b0521-d322-44cf-a763-17bb535409e6 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:27.995277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 383/773] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jun 23 09:04:27 2022] GET /identity/v3/endpoints?service_id=08675e3c7b664b9d87e17d2d7da2269e&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:28.000938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b0521-d322-44cf-a763-17bb535409e6 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:28.061564 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 391/774] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:04:28.095714 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38768df9-22d7-4dba-9dca-40944364b49b None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:28.099406 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:28.134256 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:28.241809 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38768df9-22d7-4dba-9dca-40944364b49b None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:28.247512 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b0521-d322-44cf-a763-17bb535409e6 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:28.316552 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45821e93-ce9f-44a7-98a8-f56cd5cf43e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:28.317275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45821e93-ce9f-44a7-98a8-f56cd5cf43e9 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:28.317560 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45821e93-ce9f-44a7-98a8-f56cd5cf43e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:28.317839 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45821e93-ce9f-44a7-98a8-f56cd5cf43e9 None admin] PATH_INFO: `/v3/services/network` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:28.318493 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45821e93-ce9f-44a7-98a8-f56cd5cf43e9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:28.319886 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45821e93-ce9f-44a7-98a8-f56cd5cf43e9 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:28.321386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d21c9-7dc8-41cb-98cb-34c23b86f8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:28.322044 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d21c9-7dc8-41cb-98cb-34c23b86f8b1 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:28.322327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d21c9-7dc8-41cb-98cb-34c23b86f8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:28.322603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d21c9-7dc8-41cb-98cb-34c23b86f8b1 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:28.323317 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d21c9-7dc8-41cb-98cb-34c23b86f8b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:28.324741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d21c9-7dc8-41cb-98cb-34c23b86f8b1 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:28.333674 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 384/775] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jun 23 09:04:28 2022] GET /identity/v3/services => generated 772 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:28.342892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-45821e93-ce9f-44a7-98a8-f56cd5cf43e9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 09:04:28.342892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:28.342892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:04:28.342892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:04:28.342892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:04:28.342892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:04:28.342892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:04:28.342892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:28.342892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 09:04:28.342892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:28.342892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:28.342892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:28.342892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:28.342892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:28.342892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:28.342892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:28.342892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:28.344069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:28.344069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:28.344069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:28.344069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:28.344069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:28.344069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:28.344069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:04:28.344069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:04:28.344069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:04:28.344069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:04:28.344069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:28.344069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:28.344069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:28.344069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:28.344069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:28.344069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:28.344983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:28.344983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:28.344983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:28.344983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:28.344983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:28.344983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:28.344983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:28.344983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:28.344983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:28.344983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:28.344983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:04:28.344983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:28.344983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 09:04:28.344983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:28.344983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 392/776] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:28 2022] GET /identity/v3/services/network => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:28.350225 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d21c9-7dc8-41cb-98cb-34c23b86f8b1 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:28.490859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d21c9-7dc8-41cb-98cb-34c23b86f8b1 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:28.507750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:28.544209 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb0e10-1fe5-4914-bd5a-e929f546f439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:28.544901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb0e10-1fe5-4914-bd5a-e929f546f439 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:28.545188 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb0e10-1fe5-4914-bd5a-e929f546f439 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:28.545468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb0e10-1fe5-4914-bd5a-e929f546f439 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:28.546199 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb0e10-1fe5-4914-bd5a-e929f546f439 None admin] RBAC: Adding query filter params (name=network) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:28.546507 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb0e10-1fe5-4914-bd5a-e929f546f439 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:28.547961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb0e10-1fe5-4914-bd5a-e929f546f439 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:28.555338 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 385/777] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jun 23 09:04:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:28.561077 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45821e93-ce9f-44a7-98a8-f56cd5cf43e9 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:28.643536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:28.693287 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45821e93-ce9f-44a7-98a8-f56cd5cf43e9 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:28.738450 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410495f-7da2-4888-8a9a-0653b23be286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:28.739159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410495f-7da2-4888-8a9a-0653b23be286 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:28.739449 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410495f-7da2-4888-8a9a-0653b23be286 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:28.739733 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410495f-7da2-4888-8a9a-0653b23be286 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:28.740451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410495f-7da2-4888-8a9a-0653b23be286 None admin] RBAC: Adding query filter params (type=network) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:28.740762 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410495f-7da2-4888-8a9a-0653b23be286 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:28.742133 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410495f-7da2-4888-8a9a-0653b23be286 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:28.751186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 393/778] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jun 23 09:04:28 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:28.756850 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb0e10-1fe5-4914-bd5a-e929f546f439 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:28.904842 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb0e10-1fe5-4914-bd5a-e929f546f439 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:28.954332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15860334-662d-493f-846e-392bc6363f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:28.955026 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15860334-662d-493f-846e-392bc6363f13 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:28.955316 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15860334-662d-493f-846e-392bc6363f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:28.955596 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15860334-662d-493f-846e-392bc6363f13 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:28.956466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15860334-662d-493f-846e-392bc6363f13 None admin] RBAC: Adding query filter params (interface=public, service_id=ab1dbcfc2b0743848bcf13a041efa14b, region_id=RegionOne) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:28.956784 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15860334-662d-493f-846e-392bc6363f13 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:28.958163 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15860334-662d-493f-846e-392bc6363f13 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:28.964229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 386/779] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jun 23 09:04:28 2022] GET /identity/v3/endpoints?service_id=ab1dbcfc2b0743848bcf13a041efa14b&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:28.972004 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410495f-7da2-4888-8a9a-0653b23be286 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:29.078299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410495f-7da2-4888-8a9a-0653b23be286 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:29.113708 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:29.121319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f8c06-0510-4f5b-b0ac-5285c46aea45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:29.122006 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f8c06-0510-4f5b-b0ac-5285c46aea45 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:29.122300 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f8c06-0510-4f5b-b0ac-5285c46aea45 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:29.122578 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f8c06-0510-4f5b-b0ac-5285c46aea45 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:29.123294 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f8c06-0510-4f5b-b0ac-5285c46aea45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:29.124670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f8c06-0510-4f5b-b0ac-5285c46aea45 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:29.130335 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 394/780] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jun 23 09:04:28 2022] GET /identity/v3/services => generated 772 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:29.440843 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef08a7f-7c74-4428-899b-f1ef83d7d242 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:29.441393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef08a7f-7c74-4428-899b-f1ef83d7d242 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:29.441393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef08a7f-7c74-4428-899b-f1ef83d7d242 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:29.443034 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 387/781] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:29.448293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97417e68-74b1-4b96-9a73-39b3cf073242 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:29.448587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97417e68-74b1-4b96-9a73-39b3cf073242 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:29.448870 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97417e68-74b1-4b96-9a73-39b3cf073242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:29.467492 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-97417e68-74b1-4b96-9a73-39b3cf073242 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:29.468688 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-97417e68-74b1-4b96-9a73-39b3cf073242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:29.551227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 395/782] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:04:29.581609 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef08a7f-7c74-4428-899b-f1ef83d7d242 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:29.631731 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:29.710384 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef08a7f-7c74-4428-899b-f1ef83d7d242 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:29.725295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372df324-4a90-4605-85cf-cb8d176bc300 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:29.725649 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372df324-4a90-4605-85cf-cb8d176bc300 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:29.725934 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372df324-4a90-4605-85cf-cb8d176bc300 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:29.727672 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 396/783] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:29.732787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a16d89-513d-446e-8a6f-160c6623c90b None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:29.733140 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a16d89-513d-446e-8a6f-160c6623c90b None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:29.733437 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a16d89-513d-446e-8a6f-160c6623c90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:29.744982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d863d2-5965-459b-8f59-e14eca096bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:29.745784 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d863d2-5965-459b-8f59-e14eca096bdc None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:29.746599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d863d2-5965-459b-8f59-e14eca096bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:29.746879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d863d2-5965-459b-8f59-e14eca096bdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:29.760456 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-51a16d89-513d-446e-8a6f-160c6623c90b None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:29.761305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d863d2-5965-459b-8f59-e14eca096bdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:29.761764 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-51a16d89-513d-446e-8a6f-160c6623c90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:29.770637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-e1d863d2-5965-459b-8f59-e14eca096bdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:04:29.770637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:29.770637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:29.770637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:29.770637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:29.770637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:29.770637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:29.770637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:29.770637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:29.770637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:29.770637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:29.770637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:29.770637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:29.770637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:29.770637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:29.770637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:29.771657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:29.771657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:29.771657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:29.771657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:29.771657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:29.771657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:29.771657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:29.771657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:29.771657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:29.771657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:29.771657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:29.771657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:29.771657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:29.771657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:29.771657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:29.771657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:29.772990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:29.772990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:29.772990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:29.772990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:29.772990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:29.772990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:29.772990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:29.772990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:29.772990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:29.772990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:29.772990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:04:29.772990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:29.772990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 388/784] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:04:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:29.776627 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91529c14-4dc6-43ed-89a1-7666faa4596c None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:29.776627 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91529c14-4dc6-43ed-89a1-7666faa4596c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:29.776627 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91529c14-4dc6-43ed-89a1-7666faa4596c None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:29.776627 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 389/785] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:29.782232 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d868c9-a63c-4fff-8bea-be6bda5957c3 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:29.783025 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d868c9-a63c-4fff-8bea-be6bda5957c3 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:29.783308 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d868c9-a63c-4fff-8bea-be6bda5957c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:29.805610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-18d868c9-a63c-4fff-8bea-be6bda5957c3 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:29.806806 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-18d868c9-a63c-4fff-8bea-be6bda5957c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:29.848452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 397/786] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:04:29.849485 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a16d89-513d-446e-8a6f-160c6623c90b None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:29.911082 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 390/787] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:04:29.911982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d868c9-a63c-4fff-8bea-be6bda5957c3 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:29.937571 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:29.940942 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:29.993648 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a16d89-513d-446e-8a6f-160c6623c90b None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:30.020621 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d868c9-a63c-4fff-8bea-be6bda5957c3 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:30.034253 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9c36c-d7b9-4d2e-8491-1689758889f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:30.035821 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9c36c-d7b9-4d2e-8491-1689758889f8 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:30.036116 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9c36c-d7b9-4d2e-8491-1689758889f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:30.036490 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9c36c-d7b9-4d2e-8491-1689758889f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:30.037255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9c36c-d7b9-4d2e-8491-1689758889f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:30.037567 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9c36c-d7b9-4d2e-8491-1689758889f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:30.039141 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9c36c-d7b9-4d2e-8491-1689758889f8 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:30.053597 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 398/788] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:04:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:30.054682 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9c36c-d7b9-4d2e-8491-1689758889f8 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:30.057357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78fb24-e0e7-4ce6-8129-5dd5018c7aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:30.058165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78fb24-e0e7-4ce6-8129-5dd5018c7aff None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:30.058493 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78fb24-e0e7-4ce6-8129-5dd5018c7aff None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:30.058775 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78fb24-e0e7-4ce6-8129-5dd5018c7aff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:30.069700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78fb24-e0e7-4ce6-8129-5dd5018c7aff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:30.077952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-2a78fb24-e0e7-4ce6-8129-5dd5018c7aff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:04:30.077952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:30.077952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:30.077952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:30.077952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:30.077952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:30.077952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:30.077952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:30.077952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:30.077952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:30.077952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:30.077952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:30.077952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:30.077952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:30.077952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:30.077952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:30.078988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:30.078988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:30.078988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:30.078988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:30.078988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:30.078988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:30.078988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:30.078988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:30.078988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:30.078988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:30.078988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:30.078988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:30.078988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:30.078988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:30.078988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:30.078988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:30.081051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:30.081051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:30.081051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:30.081051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:30.081051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:30.081051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:30.081051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:30.081051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:30.081051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:30.081051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:30.081051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:04:30.081051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:30.081051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 391/789] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:04:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:30.081051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78fb24-e0e7-4ce6-8129-5dd5018c7aff None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:30.173084 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:30.184037 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9c36c-d7b9-4d2e-8491-1689758889f8 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:30.184491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78fb24-e0e7-4ce6-8129-5dd5018c7aff None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:30.203167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:30.219555 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b513d5f-e39a-4ccb-a43e-37462c6e9def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:30.220359 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b513d5f-e39a-4ccb-a43e-37462c6e9def None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:30.220652 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b513d5f-e39a-4ccb-a43e-37462c6e9def None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:30.220936 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b513d5f-e39a-4ccb-a43e-37462c6e9def None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:30.223938 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d86613-3045-4130-8207-8fba822149de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:30.224898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d86613-3045-4130-8207-8fba822149de None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:30.225380 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d86613-3045-4130-8207-8fba822149de None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:30.225810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d86613-3045-4130-8207-8fba822149de None admin] PATH_INFO: `/v3/roles/service` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:30.232349 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b513d5f-e39a-4ccb-a43e-37462c6e9def None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:30.233831 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b513d5f-e39a-4ccb-a43e-37462c6e9def None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:30.237709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d86613-3045-4130-8207-8fba822149de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:30.246266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-62d86613-3045-4130-8207-8fba822149de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:30.246266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:30.246266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:30.246266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:30.246266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:30.246266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:30.246266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:30.246266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:30.246266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:30.246266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:30.246266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:30.246266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:30.246266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:30.246266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:30.246266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:30.246266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:30.247470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:30.247470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:30.247470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:30.247470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:30.247470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:30.247470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:30.247470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:30.247470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:30.247470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:30.247470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:30.247470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:30.247470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:30.247470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:30.247470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:30.247470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:30.247470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:30.248536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:30.248536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:30.248536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:30.248536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:30.248536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:30.248536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:30.248536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:30.248536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:30.248536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:30.248536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:30.248536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:30.248536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:30.255186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-9b513d5f-e39a-4ccb-a43e-37462c6e9def None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:30.255186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:30.255186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:30.255186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:30.255186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:30.255186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:30.255186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:30.255186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:30.255186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:30.255186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:30.255186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:30.255186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:30.255186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:30.255186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:30.255186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:30.256132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:30.256132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:30.256132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:30.256132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:30.256132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:30.256132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:30.256132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:30.256132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:30.256132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:30.256132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:30.256132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:30.256132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:30.256132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:30.256132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:30.256132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:30.256132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:30.256132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:30.257157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:30.257157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:30.257157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:30.257157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:30.257852 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 399/791] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:04:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:30.257952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 392/791] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:30.258733 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b513d5f-e39a-4ccb-a43e-37462c6e9def None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:30.259072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d86613-3045-4130-8207-8fba822149de None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:30.366477 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b513d5f-e39a-4ccb-a43e-37462c6e9def None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:30.380375 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d86613-3045-4130-8207-8fba822149de None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:30.405429 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803b55a-0c28-4210-9dac-05ed9fc12f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:30.406793 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803b55a-0c28-4210-9dac-05ed9fc12f48 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:30.407290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803b55a-0c28-4210-9dac-05ed9fc12f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:30.407894 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803b55a-0c28-4210-9dac-05ed9fc12f48 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:30.408916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803b55a-0c28-4210-9dac-05ed9fc12f48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:30.409531 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803b55a-0c28-4210-9dac-05ed9fc12f48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:30.411686 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803b55a-0c28-4210-9dac-05ed9fc12f48 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:30.426808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916123fd-2c0a-4625-a11f-67bbd3952d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:30.426808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916123fd-2c0a-4625-a11f-67bbd3952d56 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:30.426808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916123fd-2c0a-4625-a11f-67bbd3952d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:30.426808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916123fd-2c0a-4625-a11f-67bbd3952d56 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:30.427268 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916123fd-2c0a-4625-a11f-67bbd3952d56 None admin] RBAC: Adding query filter params (name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:30.427268 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916123fd-2c0a-4625-a11f-67bbd3952d56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:30.427268 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916123fd-2c0a-4625-a11f-67bbd3952d56 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:30.427807 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 393/792] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:04:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:30.429006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803b55a-0c28-4210-9dac-05ed9fc12f48 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:30.432470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:30.442719 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 400/793] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jun 23 09:04:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:30.442719 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916123fd-2c0a-4625-a11f-67bbd3952d56 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:30.496432 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:30.560363 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803b55a-0c28-4210-9dac-05ed9fc12f48 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:30.577996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916123fd-2c0a-4625-a11f-67bbd3952d56 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:30.597110 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715990d-319a-4764-99c7-5608a027a827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:30.598136 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715990d-319a-4764-99c7-5608a027a827 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:30.598628 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715990d-319a-4764-99c7-5608a027a827 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:30.599150 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715990d-319a-4764-99c7-5608a027a827 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:30.600082 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3715990d-319a-4764-99c7-5608a027a827 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:30.600593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3715990d-319a-4764-99c7-5608a027a827 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:30.602202 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3715990d-319a-4764-99c7-5608a027a827 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:30.615066 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 394/794] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:30.616168 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715990d-319a-4764-99c7-5608a027a827 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:30.630174 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54fc804-d6c8-4a98-9d25-cefef11afa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:30.631439 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fc804-d6c8-4a98-9d25-cefef11afa43 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:30.631735 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fc804-d6c8-4a98-9d25-cefef11afa43 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:30.632024 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fc804-d6c8-4a98-9d25-cefef11afa43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:30.644823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54fc804-d6c8-4a98-9d25-cefef11afa43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:30.646545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54fc804-d6c8-4a98-9d25-cefef11afa43 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:30.664256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-b54fc804-d6c8-4a98-9d25-cefef11afa43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:30.664256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:30.664256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:30.664256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:30.664256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:30.664256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:30.664256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:30.664256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:30.664256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:30.664256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:30.664256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:30.664256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:30.664256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:30.664256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:30.664256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:30.665218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:30.665218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:30.665218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:30.665218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:30.665218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:30.665218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:30.665218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:30.665218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:30.665218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:30.665218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:30.665218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:30.665218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:30.665218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:30.665218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:30.665218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:30.665218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:30.665218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:30.666180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:30.666180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:30.666180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:30.666180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:30.672971 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 401/795] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:30.674018 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54fc804-d6c8-4a98-9d25-cefef11afa43 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:30.692148 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:30.744154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715990d-319a-4764-99c7-5608a027a827 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:30.774262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:30.783595 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54fc804-d6c8-4a98-9d25-cefef11afa43 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:30.784620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0004380-a8d0-4d8a-ae49-e4f46010a03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:30.785521 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0004380-a8d0-4d8a-ae49-e4f46010a03c None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:30.785764 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0004380-a8d0-4d8a-ae49-e4f46010a03c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:30.786047 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0004380-a8d0-4d8a-ae49-e4f46010a03c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:30.797358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0004380-a8d0-4d8a-ae49-e4f46010a03c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:30.798842 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0004380-a8d0-4d8a-ae49-e4f46010a03c None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:30.817697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-a0004380-a8d0-4d8a-ae49-e4f46010a03c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:30.817697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:30.817697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:30.817697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:30.817697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:30.817697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:30.817697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:30.817697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:30.817697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:30.817697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:30.817697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:30.817697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:30.817697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:30.817697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:30.817697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:30.818722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:30.818722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:30.818722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:30.818722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:30.818722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:30.818722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:30.818722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:30.818722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:30.818722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:30.818722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:30.818722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:30.818722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:30.818722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:30.818722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:30.818722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:30.818722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:30.818722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:30.819706 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:30.819706 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:30.819706 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:30.819706 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:30.823059 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 395/796] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:30.824691 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0004380-a8d0-4d8a-ae49-e4f46010a03c None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:30.827897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43c1b5-edb2-4b4b-827e-4830c1be03c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:30.829039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43c1b5-edb2-4b4b-827e-4830c1be03c4 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:30.829331 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43c1b5-edb2-4b4b-827e-4830c1be03c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:30.829612 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43c1b5-edb2-4b4b-827e-4830c1be03c4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:30.838504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc43c1b5-edb2-4b4b-827e-4830c1be03c4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:30.840425 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc43c1b5-edb2-4b4b-827e-4830c1be03c4 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:30.859596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-fc43c1b5-edb2-4b4b-827e-4830c1be03c4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 09:04:30.859596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:30.859596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:30.859596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:30.859596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:30.859596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:30.859596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:30.859596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:30.859596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:30.859596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:30.859596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:30.859596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:30.859596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:30.859596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:30.859596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:30.859596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:30.860662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:30.860662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:30.860662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:30.860662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:30.860662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:30.860662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:30.860662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:30.860662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:30.860662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:30.860662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:30.860662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:30.860662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:30.860662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:30.860662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:30.860662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:30.860662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:30.860662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:30.860662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:30.861694 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:30.861694 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:30.861694 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:30.861694 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:30.861694 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:30.861694 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:30.861694 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:30.861694 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:30.861694 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 09:04:30.861694 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:30.864559 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 402/797] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:04:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:30.865578 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43c1b5-edb2-4b4b-827e-4830c1be03c4 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:30.945291 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0004380-a8d0-4d8a-ae49-e4f46010a03c None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:30.961128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:30.977490 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43c1b5-edb2-4b4b-827e-4830c1be03c4 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:30.984512 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b56f1-fe0d-44f8-a1c8-c88ff689ba5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:30.985201 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b56f1-fe0d-44f8-a1c8-c88ff689ba5b None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:30.985543 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b56f1-fe0d-44f8-a1c8-c88ff689ba5b None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:30.985827 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b56f1-fe0d-44f8-a1c8-c88ff689ba5b None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:30.986561 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b56f1-fe0d-44f8-a1c8-c88ff689ba5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:30.986875 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b56f1-fe0d-44f8-a1c8-c88ff689ba5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:30.988301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b56f1-fe0d-44f8-a1c8-c88ff689ba5b None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:31.005434 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 396/798] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:31.005434 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b56f1-fe0d-44f8-a1c8-c88ff689ba5b None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:31.017706 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c889a8-6421-48b9-8bc8-2dfb91169332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:31.018381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c889a8-6421-48b9-8bc8-2dfb91169332 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:31.018669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c889a8-6421-48b9-8bc8-2dfb91169332 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:31.018979 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c889a8-6421-48b9-8bc8-2dfb91169332 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:31.019718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c889a8-6421-48b9-8bc8-2dfb91169332 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:31.020027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c889a8-6421-48b9-8bc8-2dfb91169332 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:31.021391 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c889a8-6421-48b9-8bc8-2dfb91169332 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:31.038106 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 403/799] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:31.038106 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a9d23-3464-442d-a28e-11aa745d965f None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:31.038106 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a9d23-3464-442d-a28e-11aa745d965f None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:31.038106 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a9d23-3464-442d-a28e-11aa745d965f None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:31.038106 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 404/800] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:31.038106 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a9d23-3464-442d-a28e-11aa745d965f None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:31.055012 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:31.118786 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b56f1-fe0d-44f8-a1c8-c88ff689ba5b None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:31.160072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a9d23-3464-442d-a28e-11aa745d965f None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:31.179924 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d76f36-7ff2-43d7-b30f-d82cfa492d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:31.180609 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d76f36-7ff2-43d7-b30f-d82cfa492d66 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:31.180929 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d76f36-7ff2-43d7-b30f-d82cfa492d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:31.181196 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d76f36-7ff2-43d7-b30f-d82cfa492d66 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:31.182005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d76f36-7ff2-43d7-b30f-d82cfa492d66 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:31.182448 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d76f36-7ff2-43d7-b30f-d82cfa492d66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:31.183792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d76f36-7ff2-43d7-b30f-d82cfa492d66 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:31.200901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e97b9-c622-4910-b6a9-5f7f63e0e79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:31.201701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e97b9-c622-4910-b6a9-5f7f63e0e79d None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:31.201988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e97b9-c622-4910-b6a9-5f7f63e0e79d None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:31.202339 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e97b9-c622-4910-b6a9-5f7f63e0e79d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:31.205814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 397/801] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jun 23 09:04:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:31.206692 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d76f36-7ff2-43d7-b30f-d82cfa492d66 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:31.210358 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e97b9-c622-4910-b6a9-5f7f63e0e79d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:31.211827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e97b9-c622-4910-b6a9-5f7f63e0e79d None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:31.227228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-f48e97b9-c622-4910-b6a9-5f7f63e0e79d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 09:04:31.227228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:31.227228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:31.227228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:31.227228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:31.227228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:31.227228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:31.227228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:31.227228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:31.227228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:31.227228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:31.227228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:31.227228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:31.227228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:31.227228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:31.227228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:31.228292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:31.228292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:31.228292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:31.228292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:31.228292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:31.228292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:31.228292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:31.228292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:31.228292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:31.228292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:31.228292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:31.228292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:31.228292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:31.228292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:31.228292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:31.228292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:31.228292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:31.228292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:31.229301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:31.229301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:31.229301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:31.229301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:31.229301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:31.229301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:31.229301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:31.229301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:31.229301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 09:04:31.229301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:31.229301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 405/802] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jun 23 09:04:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:31.229957 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79967c99-eb52-461e-ab88-2da5b598a19d None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:31.231367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79967c99-eb52-461e-ab88-2da5b598a19d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:31.231367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79967c99-eb52-461e-ab88-2da5b598a19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:31.246836 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:31.247597 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-79967c99-eb52-461e-ab88-2da5b598a19d None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:31.248952 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-79967c99-eb52-461e-ab88-2da5b598a19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:31.318957 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 406/803] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:04:31.319181 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d76f36-7ff2-43d7-b30f-d82cfa492d66 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:31.319281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79967c99-eb52-461e-ab88-2da5b598a19d None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:31.320501 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:31.357819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db639a14-8cc6-4eaf-9c5b-33facd524bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:31.358945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db639a14-8cc6-4eaf-9c5b-33facd524bbc None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:31.359409 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db639a14-8cc6-4eaf-9c5b-33facd524bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:31.359870 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db639a14-8cc6-4eaf-9c5b-33facd524bbc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:31.367385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db639a14-8cc6-4eaf-9c5b-33facd524bbc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:31.368881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db639a14-8cc6-4eaf-9c5b-33facd524bbc None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:31.384801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-db639a14-8cc6-4eaf-9c5b-33facd524bbc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 09:04:31.384801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:31.384801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:31.384801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:31.384801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:31.384801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:31.384801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:31.384801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:31.384801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:31.384801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:31.384801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:31.384801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:31.384801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:31.384801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:31.384801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:31.384801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:31.385811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:31.385811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:31.385811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:31.385811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:31.385811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:31.385811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:31.385811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:31.385811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:31.385811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:31.385811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:31.385811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:31.385811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:31.385811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:31.385811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:31.385811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:31.385811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:31.385811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:31.385811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:31.386891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:31.386891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:31.386891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:31.386891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:31.386891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:31.386891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:31.386891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:31.386891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:31.386891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 09:04:31.386891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:31.386891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 398/804] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jun 23 09:04:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:31.386891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db639a14-8cc6-4eaf-9c5b-33facd524bbc None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:31.427806 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79967c99-eb52-461e-ab88-2da5b598a19d None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:31.463125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0718f-cf4d-4229-bb87-fd8815e137cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:31.463925 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0718f-cf4d-4229-bb87-fd8815e137cf None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:31.464213 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0718f-cf4d-4229-bb87-fd8815e137cf None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:31.464493 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0718f-cf4d-4229-bb87-fd8815e137cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:31.474787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0718f-cf4d-4229-bb87-fd8815e137cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:31.476268 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0718f-cf4d-4229-bb87-fd8815e137cf None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:31.482493 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:31.490881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db639a14-8cc6-4eaf-9c5b-33facd524bbc None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:31.494039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-a2c0718f-cf4d-4229-bb87-fd8815e137cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:31.494039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:31.494039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:31.494039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:31.494039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:31.494039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:31.494039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:31.494039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:31.494039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:31.494039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:31.494039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:31.494039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:31.494039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:31.494039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:31.494039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:31.499557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:31.499557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:31.499557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:31.499557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:31.499557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:31.499557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:31.499557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:31.499557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:31.499557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:31.499557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:31.499557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:31.499557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:31.499557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:31.499557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:31.499557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:31.499557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:31.499557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:31.500982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:31.500982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:31.500982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:31.500982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:31.500982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 407/805] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:31.500982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0718f-cf4d-4229-bb87-fd8815e137cf None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:31.531959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca409f-3a2e-48cd-b54c-d3e77f3d1d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:31.532633 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca409f-3a2e-48cd-b54c-d3e77f3d1d2d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:31.532918 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca409f-3a2e-48cd-b54c-d3e77f3d1d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:31.533192 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca409f-3a2e-48cd-b54c-d3e77f3d1d2d None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:31.534027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca409f-3a2e-48cd-b54c-d3e77f3d1d2d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:31.534337 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca409f-3a2e-48cd-b54c-d3e77f3d1d2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:31.536094 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca409f-3a2e-48cd-b54c-d3e77f3d1d2d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:31.554625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 399/806] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jun 23 09:04:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:31.555482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca409f-3a2e-48cd-b54c-d3e77f3d1d2d None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:31.574994 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:31.618717 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0718f-cf4d-4229-bb87-fd8815e137cf None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:31.651189 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af640d-f4b2-454e-b783-faabcbc52f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:31.651995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af640d-f4b2-454e-b783-faabcbc52f06 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:31.652382 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af640d-f4b2-454e-b783-faabcbc52f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:31.652660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af640d-f4b2-454e-b783-faabcbc52f06 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:31.653312 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af640d-f4b2-454e-b783-faabcbc52f06 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:31.654687 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af640d-f4b2-454e-b783-faabcbc52f06 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:31.657732 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca409f-3a2e-48cd-b54c-d3e77f3d1d2d None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:31.666615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-d1af640d-f4b2-454e-b783-faabcbc52f06 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 09:04:31.666615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:31.666615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:04:31.666615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:04:31.666615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:04:31.666615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:04:31.666615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:04:31.666615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:31.666615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 09:04:31.666615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:31.666615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:31.666615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:31.666615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:31.666615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:31.666615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:31.666615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:31.673227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:31.673227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:31.673227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:31.673227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:31.673227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:31.673227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:31.673227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:31.673227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:04:31.673227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:04:31.673227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:04:31.673227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:04:31.673227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:31.673227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:31.673227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:31.673227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:31.673227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:31.674395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:31.674395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:31.674395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:31.674395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:31.674395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:31.674395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:31.674395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:31.674395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:31.674395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:31.674395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:31.674395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:31.674395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:04:31.674395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:31.674395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 09:04:31.674395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:31.674395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 408/807] 10.222.0.52 () {56 vars in 1129 bytes} [Thu Jun 23 09:04:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:31.675835 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34c5c2-60fa-4d4b-a340-2639809234c1 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:31.675835 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34c5c2-60fa-4d4b-a340-2639809234c1 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:31.675835 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34c5c2-60fa-4d4b-a340-2639809234c1 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:31.675835 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 409/808] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:31.675835 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34c5c2-60fa-4d4b-a340-2639809234c1 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:31.695822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4a9e9-6a0f-4222-9898-4565af18f9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:31.696511 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4a9e9-6a0f-4222-9898-4565af18f9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:31.696793 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4a9e9-6a0f-4222-9898-4565af18f9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:31.697065 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4a9e9-6a0f-4222-9898-4565af18f9e0 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:31.697918 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd4a9e9-6a0f-4222-9898-4565af18f9e0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:31.698228 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd4a9e9-6a0f-4222-9898-4565af18f9e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:31.699774 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd4a9e9-6a0f-4222-9898-4565af18f9e0 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:31.719656 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 400/809] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jun 23 09:04:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:31.720304 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4a9e9-6a0f-4222-9898-4565af18f9e0 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:31.733397 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:31.822582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4a9e9-6a0f-4222-9898-4565af18f9e0 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:31.872131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc95ee-e45e-48d1-8c60-a54d475e1168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:31.872703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc95ee-e45e-48d1-8c60-a54d475e1168 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:31.872990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc95ee-e45e-48d1-8c60-a54d475e1168 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:31.873267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc95ee-e45e-48d1-8c60-a54d475e1168 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:31.890529 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34c5c2-60fa-4d4b-a340-2639809234c1 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:31.907232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:31.923214 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aaf32f-83e6-4571-af82-3e67266e31a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:31.923877 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aaf32f-83e6-4571-af82-3e67266e31a3 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:31.924168 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aaf32f-83e6-4571-af82-3e67266e31a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:31.924446 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aaf32f-83e6-4571-af82-3e67266e31a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:31.925166 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aaf32f-83e6-4571-af82-3e67266e31a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:31.925475 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aaf32f-83e6-4571-af82-3e67266e31a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:31.926845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aaf32f-83e6-4571-af82-3e67266e31a3 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:31.943042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 410/810] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:31.943042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aaf32f-83e6-4571-af82-3e67266e31a3 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:31.943311 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc95ee-e45e-48d1-8c60-a54d475e1168 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:31.943553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc95ee-e45e-48d1-8c60-a54d475e1168 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:31.953289 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:32.038007 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aaf32f-83e6-4571-af82-3e67266e31a3 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:32.049807 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 401/811] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:04:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 09:04:32.051267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3e05d-2897-460c-a387-926b668d5d43 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:32.051614 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3e05d-2897-460c-a387-926b668d5d43 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:32.051896 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3e05d-2897-460c-a387-926b668d5d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:32.068095 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-eeb3e05d-2897-460c-a387-926b668d5d43 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:32.069325 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-eeb3e05d-2897-460c-a387-926b668d5d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:32.075109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61165071-9ae5-4a6c-91ac-42ce91f1aa50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:32.075883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61165071-9ae5-4a6c-91ac-42ce91f1aa50 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:32.076221 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61165071-9ae5-4a6c-91ac-42ce91f1aa50 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:32.076506 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61165071-9ae5-4a6c-91ac-42ce91f1aa50 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:32.077304 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61165071-9ae5-4a6c-91ac-42ce91f1aa50 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:32.077613 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61165071-9ae5-4a6c-91ac-42ce91f1aa50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:32.079016 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61165071-9ae5-4a6c-91ac-42ce91f1aa50 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:32.083411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 411/812] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jun 23 09:04:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:32.084432 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61165071-9ae5-4a6c-91ac-42ce91f1aa50 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:32.126172 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:32.147307 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 402/813] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:04:32.147307 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3e05d-2897-460c-a387-926b668d5d43 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:32.186069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61165071-9ae5-4a6c-91ac-42ce91f1aa50 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:32.188231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:32.219201 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd0932-f832-43ae-a814-9eda95822c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:32.219998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd0932-f832-43ae-a814-9eda95822c7e None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:32.220283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd0932-f832-43ae-a814-9eda95822c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:32.220558 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd0932-f832-43ae-a814-9eda95822c7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:32.231181 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd0932-f832-43ae-a814-9eda95822c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:32.232610 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd0932-f832-43ae-a814-9eda95822c7e None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:32.247125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3e05d-2897-460c-a387-926b668d5d43 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:32.249251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-9ebd0932-f832-43ae-a814-9eda95822c7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:32.249251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:32.249251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:32.249251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:32.249251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:32.249251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:32.249251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:32.249251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:32.249251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:32.249251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:32.249251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:32.249251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:32.249251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:32.249251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:32.249251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:32.250350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:32.250350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:32.250350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:32.250350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:32.250350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:32.250350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:32.250350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:32.250350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:32.250350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:32.250350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:32.250350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:32.250350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:32.250350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:32.250350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:32.250350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:32.250350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:32.250350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:32.251385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:32.251385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:32.251385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:32.251385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:32.251385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 412/814] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:32.251385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd0932-f832-43ae-a814-9eda95822c7e None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:32.281851 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13244213-8293-4136-958a-e23e13ba00c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:32.282645 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13244213-8293-4136-958a-e23e13ba00c8 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:32.282971 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13244213-8293-4136-958a-e23e13ba00c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:32.283253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13244213-8293-4136-958a-e23e13ba00c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:32.289911 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13244213-8293-4136-958a-e23e13ba00c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:32.291413 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13244213-8293-4136-958a-e23e13ba00c8 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:32.305487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-13244213-8293-4136-958a-e23e13ba00c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:32.305487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:32.305487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:32.305487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:32.305487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:32.305487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:32.305487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:32.305487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:32.305487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:32.305487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:32.305487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:32.305487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:32.305487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:32.305487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:32.305487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:32.306430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:32.306430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:32.306430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:32.306430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:32.306430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:32.306430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:32.306430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:32.306430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:32.306430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:32.306430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:32.306430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:32.306430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:32.306430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:32.306430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:32.306430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:32.306430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:32.306430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:32.307435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:32.307435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:32.307435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:32.307435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:32.307435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:32.307435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:32.307435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:32.307435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:32.307435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 403/815] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:32.307962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13244213-8293-4136-958a-e23e13ba00c8 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:32.347688 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:32.354379 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd0932-f832-43ae-a814-9eda95822c7e None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:32.387877 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09de2a9-78f9-482c-906e-f138ca3488a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:32.388650 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09de2a9-78f9-482c-906e-f138ca3488a5 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:32.389303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09de2a9-78f9-482c-906e-f138ca3488a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:32.389587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09de2a9-78f9-482c-906e-f138ca3488a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:32.399644 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09de2a9-78f9-482c-906e-f138ca3488a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:32.401100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09de2a9-78f9-482c-906e-f138ca3488a5 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:32.403155 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:32.406807 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13244213-8293-4136-958a-e23e13ba00c8 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:32.419635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-f09de2a9-78f9-482c-906e-f138ca3488a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:32.419635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:32.419635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:32.419635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:32.419635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:32.419635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:32.419635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:32.419635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:32.419635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:32.419635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:32.419635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:32.419635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:32.419635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:32.419635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:32.419635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:32.420674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:32.420674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:32.420674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:32.420674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:32.420674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:32.420674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:32.420674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:32.420674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:32.420674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:32.420674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:32.420674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:32.420674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:32.420674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:32.420674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:32.420674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:32.420674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:32.420674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:32.421637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:32.421637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:32.421637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:32.421637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:32.421637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 413/816] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:32.421949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09de2a9-78f9-482c-906e-f138ca3488a5 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:32.443596 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102fcf3-0d13-4e2f-97b8-78a71c467ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:32.444258 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102fcf3-0d13-4e2f-97b8-78a71c467ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:32.444547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102fcf3-0d13-4e2f-97b8-78a71c467ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:32.444824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102fcf3-0d13-4e2f-97b8-78a71c467ca4 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:32.445532 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102fcf3-0d13-4e2f-97b8-78a71c467ca4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:32.445840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102fcf3-0d13-4e2f-97b8-78a71c467ca4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:32.447248 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102fcf3-0d13-4e2f-97b8-78a71c467ca4 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:32.453074 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 404/817] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jun 23 09:04:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:32.453986 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102fcf3-0d13-4e2f-97b8-78a71c467ca4 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:32.518705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09de2a9-78f9-482c-906e-f138ca3488a5 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:32.551916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102fcf3-0d13-4e2f-97b8-78a71c467ca4 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:32.552076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99454b34-c606-4830-a96e-d4159d32517c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:32.552829 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99454b34-c606-4830-a96e-d4159d32517c None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:32.553113 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99454b34-c606-4830-a96e-d4159d32517c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:32.553389 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99454b34-c606-4830-a96e-d4159d32517c None admin] PATH_INFO: `/v3/services/network` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:32.554035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99454b34-c606-4830-a96e-d4159d32517c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:32.555438 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99454b34-c606-4830-a96e-d4159d32517c None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:32.568186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-99454b34-c606-4830-a96e-d4159d32517c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 09:04:32.568186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:32.568186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:04:32.568186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:04:32.568186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:04:32.568186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:04:32.568186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:04:32.568186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:32.568186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 09:04:32.568186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:32.568186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:32.568186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:32.568186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:32.568186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:32.568186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:32.568186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:32.568186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:32.569262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:32.569262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:32.569262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:32.569262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:32.569262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:32.569262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:32.569262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:04:32.569262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:04:32.569262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:04:32.569262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:04:32.569262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:32.569262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:32.569262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:32.569262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:32.569262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:32.569262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:32.570170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:32.570170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:32.570170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:32.570170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:32.570170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:32.570170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:32.570170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:32.570170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:32.570170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:32.570170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:32.570170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:04:32.570170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:32.570170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 09:04:32.570170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:32.570170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 414/818] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:32 2022] GET /identity/v3/services/network => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:32.571165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99454b34-c606-4830-a96e-d4159d32517c None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:32.583188 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:32.592763 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981c96c-b3bd-48c3-b754-8826ca000ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:32.593436 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981c96c-b3bd-48c3-b754-8826ca000ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:32.593724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981c96c-b3bd-48c3-b754-8826ca000ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:32.595922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981c96c-b3bd-48c3-b754-8826ca000ff6 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:32.595922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981c96c-b3bd-48c3-b754-8826ca000ff6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:32.595922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981c96c-b3bd-48c3-b754-8826ca000ff6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:32.596504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981c96c-b3bd-48c3-b754-8826ca000ff6 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:32.609616 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 405/819] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:32.610418 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981c96c-b3bd-48c3-b754-8826ca000ff6 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:32.636087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:32.671504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99454b34-c606-4830-a96e-d4159d32517c None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:32.704738 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7fee21-b8a5-412c-84f0-5ed1fe742906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:32.705402 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fee21-b8a5-412c-84f0-5ed1fe742906 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:32.705692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fee21-b8a5-412c-84f0-5ed1fe742906 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:32.705973 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fee21-b8a5-412c-84f0-5ed1fe742906 None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:32.706767 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7fee21-b8a5-412c-84f0-5ed1fe742906 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:32.707114 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7fee21-b8a5-412c-84f0-5ed1fe742906 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:32.708552 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7fee21-b8a5-412c-84f0-5ed1fe742906 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:32.712701 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981c96c-b3bd-48c3-b754-8826ca000ff6 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:32.720998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 415/820] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:32.721770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7fee21-b8a5-412c-84f0-5ed1fe742906 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:32.752210 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef12ae-6e5a-438e-958a-5c52ea17f4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:32.752875 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef12ae-6e5a-438e-958a-5c52ea17f4ae None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:32.753166 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef12ae-6e5a-438e-958a-5c52ea17f4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:32.753445 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef12ae-6e5a-438e-958a-5c52ea17f4ae None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:32.754172 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef12ae-6e5a-438e-958a-5c52ea17f4ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:32.754482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef12ae-6e5a-438e-958a-5c52ea17f4ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:32.755891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef12ae-6e5a-438e-958a-5c52ea17f4ae None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:32.771205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 406/821] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:32.771205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef12ae-6e5a-438e-958a-5c52ea17f4ae None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:32.788074 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:32.817195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7fee21-b8a5-412c-84f0-5ed1fe742906 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:32.848348 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:32.851453 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18f575-a631-4ae0-bcfd-9669c6b2f461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:32.852101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18f575-a631-4ae0-bcfd-9669c6b2f461 None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:32.852390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18f575-a631-4ae0-bcfd-9669c6b2f461 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:32.852731 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18f575-a631-4ae0-bcfd-9669c6b2f461 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:32.853480 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18f575-a631-4ae0-bcfd-9669c6b2f461 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:32.854897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18f575-a631-4ae0-bcfd-9669c6b2f461 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:32.866184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef12ae-6e5a-438e-958a-5c52ea17f4ae None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:32.900090 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a17c4-dca0-41a9-a6c1-d22b547700a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:32.900751 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a17c4-dca0-41a9-a6c1-d22b547700a7 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:32.902140 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a17c4-dca0-41a9-a6c1-d22b547700a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:32.902140 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a17c4-dca0-41a9-a6c1-d22b547700a7 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:32.902140 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a17c4-dca0-41a9-a6c1-d22b547700a7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:32.902445 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a17c4-dca0-41a9-a6c1-d22b547700a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:32.903692 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a17c4-dca0-41a9-a6c1-d22b547700a7 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:32.908058 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 407/822] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jun 23 09:04:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:32.908819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a17c4-dca0-41a9-a6c1-d22b547700a7 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:32.983061 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 416/823] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jun 23 09:04:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:04:32.983061 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18f575-a631-4ae0-bcfd-9669c6b2f461 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:33.011689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a17c4-dca0-41a9-a6c1-d22b547700a7 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:33.053331 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6611e-e2d4-41bf-85df-1ae0c5ec3cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:33.054134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6611e-e2d4-41bf-85df-1ae0c5ec3cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:33.054419 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6611e-e2d4-41bf-85df-1ae0c5ec3cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:33.054699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6611e-e2d4-41bf-85df-1ae0c5ec3cf7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:33.063027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6611e-e2d4-41bf-85df-1ae0c5ec3cf7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:33.064505 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6611e-e2d4-41bf-85df-1ae0c5ec3cf7 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:33.079583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-3fc6611e-e2d4-41bf-85df-1ae0c5ec3cf7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:33.079583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:33.079583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:33.079583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:33.079583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:33.079583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:33.079583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:33.079583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:33.079583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:33.079583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:33.079583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:33.079583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:33.079583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:33.079583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:33.079583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:33.080840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:33.080840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:33.080840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:33.080840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:33.080840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:33.080840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:33.080840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:33.080840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:33.080840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:33.080840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:33.080840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:33.080840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:33.080840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:33.080840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:33.080840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:33.080840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:33.080840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:33.081810 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:33.081810 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:33.081810 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:33.081810 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:33.081810 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:33.081810 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:33.081810 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:33.081810 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:33.081810 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 408/824] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:33.082325 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6611e-e2d4-41bf-85df-1ae0c5ec3cf7 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:33.085466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:33.099595 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18f575-a631-4ae0-bcfd-9669c6b2f461 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:33.131765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:33.141417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b6adca-83c6-486f-af75-c6d1b397e428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:33.142318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b6adca-83c6-486f-af75-c6d1b397e428 None admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:33.142599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b6adca-83c6-486f-af75-c6d1b397e428 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:33.142878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b6adca-83c6-486f-af75-c6d1b397e428 None admin] PATH_INFO: `/v3/projects/93b5040533524222b126bbe93afc56d1/users/5b1331b4f0924bb8b137a22a33c9d130/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:33.144862 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b6adca-83c6-486f-af75-c6d1b397e428 None admin] RBAC: Authorizing `identity:create_grant(project_id=93b5040533524222b126bbe93afc56d1, user_id=5b1331b4f0924bb8b137a22a33c9d130, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:33.146294 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b6adca-83c6-486f-af75-c6d1b397e428 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:33.193431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6611e-e2d4-41bf-85df-1ae0c5ec3cf7 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:33.228173 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a334be0-87c3-4c00-91cd-ca722a6fa267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:33.228968 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a334be0-87c3-4c00-91cd-ca722a6fa267 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:33.229256 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a334be0-87c3-4c00-91cd-ca722a6fa267 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:33.229535 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a334be0-87c3-4c00-91cd-ca722a6fa267 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:33.241268 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a334be0-87c3-4c00-91cd-ca722a6fa267 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:33.241268 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a334be0-87c3-4c00-91cd-ca722a6fa267 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:33.251426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-9a334be0-87c3-4c00-91cd-ca722a6fa267 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:33.251426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:33.251426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:33.251426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:33.251426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:33.251426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:33.251426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:33.251426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:33.251426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:33.251426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:33.251426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:33.251426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:33.251426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:33.251426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:33.251426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:33.252393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:33.252393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:33.252393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:33.252393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:33.252393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:33.252393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:33.252393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:33.252393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:33.252393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:33.252393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:33.252393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:33.252393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:33.252393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:33.252393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:33.252393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:33.252393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:33.252393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:33.253354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:33.253354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:33.253354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:33.253354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:33.253354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:33.253354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:33.253354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:33.253354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:33.253354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 409/825] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:33.253354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a334be0-87c3-4c00-91cd-ca722a6fa267 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:33.271280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 417/826] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:04:32 2022] PUT /identity/v3/projects/93b5040533524222b126bbe93afc56d1/users/5b1331b4f0924bb8b137a22a33c9d130/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:04:33.271280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b6adca-83c6-486f-af75-c6d1b397e428 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:33.333007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:33.376944 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a334be0-87c3-4c00-91cd-ca722a6fa267 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:33.418614 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289ea7b-1a6a-4704-91eb-a96c5f08cc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:33.418614 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289ea7b-1a6a-4704-91eb-a96c5f08cc74 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:33.418958 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289ea7b-1a6a-4704-91eb-a96c5f08cc74 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:33.419315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289ea7b-1a6a-4704-91eb-a96c5f08cc74 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:33.420049 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289ea7b-1a6a-4704-91eb-a96c5f08cc74 None admin] RBAC: Adding query filter params (type=network) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:33.420363 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289ea7b-1a6a-4704-91eb-a96c5f08cc74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:33.420734 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b6adca-83c6-486f-af75-c6d1b397e428 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:33.423058 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289ea7b-1a6a-4704-91eb-a96c5f08cc74 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:33.434458 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 410/827] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jun 23 09:04:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:33.436107 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289ea7b-1a6a-4704-91eb-a96c5f08cc74 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:33.470592 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb35bc-d5d4-45b0-aca6-f0e7723c27f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:33.471375 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb35bc-d5d4-45b0-aca6-f0e7723c27f6 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:33.471894 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb35bc-d5d4-45b0-aca6-f0e7723c27f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:33.472189 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb35bc-d5d4-45b0-aca6-f0e7723c27f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:33.473741 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bb35bc-d5d4-45b0-aca6-f0e7723c27f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:33.474063 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bb35bc-d5d4-45b0-aca6-f0e7723c27f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:33.475819 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bb35bc-d5d4-45b0-aca6-f0e7723c27f6 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:33.494261 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 418/828] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:33.495052 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb35bc-d5d4-45b0-aca6-f0e7723c27f6 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:33.552671 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:33.613448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb35bc-d5d4-45b0-aca6-f0e7723c27f6 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:33.654663 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8eeec-c76e-4b37-9a7d-d55f9d96a960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:33.655376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8eeec-c76e-4b37-9a7d-d55f9d96a960 None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:33.655668 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8eeec-c76e-4b37-9a7d-d55f9d96a960 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:33.655955 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8eeec-c76e-4b37-9a7d-d55f9d96a960 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:33.656709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8eeec-c76e-4b37-9a7d-d55f9d96a960 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:33.658644 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8eeec-c76e-4b37-9a7d-d55f9d96a960 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:33.670437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289ea7b-1a6a-4704-91eb-a96c5f08cc74 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:33.697676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:33.706627 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0c78c-9ef6-4ff9-a759-391ff10dd0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:33.707397 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0c78c-9ef6-4ff9-a759-391ff10dd0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:33.707773 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0c78c-9ef6-4ff9-a759-391ff10dd0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:33.708124 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0c78c-9ef6-4ff9-a759-391ff10dd0c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:33.709308 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0c78c-9ef6-4ff9-a759-391ff10dd0c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:33.709679 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0c78c-9ef6-4ff9-a759-391ff10dd0c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:33.711155 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0c78c-9ef6-4ff9-a759-391ff10dd0c0 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:33.728695 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 411/829] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:33.728695 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0c78c-9ef6-4ff9-a759-391ff10dd0c0 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:33.762038 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 419/830] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jun 23 09:04:33 2022] POST /identity/v3/endpoints => generated 327 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:04:33.763990 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8eeec-c76e-4b37-9a7d-d55f9d96a960 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:33.905451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8eeec-c76e-4b37-9a7d-d55f9d96a960 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:33.921166 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0c78c-9ef6-4ff9-a759-391ff10dd0c0 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:33.931278 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:33.947745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2008cd4-2953-42f6-958a-4939a1fc6c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:33.948898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2008cd4-2953-42f6-958a-4939a1fc6c75 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:33.949610 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2008cd4-2953-42f6-958a-4939a1fc6c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:33.950107 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2008cd4-2953-42f6-958a-4939a1fc6c75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:33.951184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2008cd4-2953-42f6-958a-4939a1fc6c75 None admin] RBAC: Adding query filter params (role.id=9cd080b50c6f4ee8a54bf9ef1655bd83, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=ff025cb706d243778468ae4570374690) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:33.951726 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2008cd4-2953-42f6-958a-4939a1fc6c75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:33.953372 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2008cd4-2953-42f6-958a-4939a1fc6c75 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:33.961076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 420/831] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:04:33 2022] GET /identity/v3/role_assignments?user.id=ff025cb706d243778468ae4570374690&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:33.987626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61092f1-2e28-45dd-a736-0d9c18b043b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:33.988549 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61092f1-2e28-45dd-a736-0d9c18b043b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:33.988838 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61092f1-2e28-45dd-a736-0d9c18b043b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:33.989208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61092f1-2e28-45dd-a736-0d9c18b043b1 None admin] PATH_INFO: `/v3/projects/93b5040533524222b126bbe93afc56d1/users/cfbaba84402d4c31bafcd12133d74c03/roles/edf696ae763147cab68434524d1eff9c` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:34.007601 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61092f1-2e28-45dd-a736-0d9c18b043b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=93b5040533524222b126bbe93afc56d1, user_id=cfbaba84402d4c31bafcd12133d74c03, role_id=edf696ae763147cab68434524d1eff9c)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:34.010821 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61092f1-2e28-45dd-a736-0d9c18b043b1 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:34.116263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 412/832] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:04:33 2022] PUT /identity/v3/projects/93b5040533524222b126bbe93afc56d1/users/cfbaba84402d4c31bafcd12133d74c03/roles/edf696ae763147cab68434524d1eff9c => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:04:35.079147 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb633cc-cea5-4dcb-bb54-175788a307ae None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:35.079669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb633cc-cea5-4dcb-bb54-175788a307ae None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:35.079669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb633cc-cea5-4dcb-bb54-175788a307ae None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:35.081357 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 421/833] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:35.087246 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99bc57-3a57-4eb5-b705-35da8e353f14 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:35.087676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99bc57-3a57-4eb5-b705-35da8e353f14 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:35.087861 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99bc57-3a57-4eb5-b705-35da8e353f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:35.105629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-3a99bc57-3a57-4eb5-b705-35da8e353f14 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:35.106826 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-3a99bc57-3a57-4eb5-b705-35da8e353f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:35.137457 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:35.177711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 413/834] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:04:35.200571 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb633cc-cea5-4dcb-bb54-175788a307ae None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:35.319427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb633cc-cea5-4dcb-bb54-175788a307ae None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:35.355933 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134da83-c4b6-4575-833c-e7cbd9ba531b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:35.356660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134da83-c4b6-4575-833c-e7cbd9ba531b None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:35.356950 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134da83-c4b6-4575-833c-e7cbd9ba531b None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:35.357234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134da83-c4b6-4575-833c-e7cbd9ba531b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:35.357889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134da83-c4b6-4575-833c-e7cbd9ba531b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:35.359290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134da83-c4b6-4575-833c-e7cbd9ba531b None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:35.371271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-2134da83-c4b6-4575-833c-e7cbd9ba531b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 09:04:35.371271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:35.371271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:04:35.371271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:04:35.371271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:04:35.371271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:04:35.371271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:04:35.371271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:35.371271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 09:04:35.371271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:35.371271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:35.371271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:35.371271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:35.371271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:35.371271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:35.371271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:35.371271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:35.372413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:35.372413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:35.372413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:35.372413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:35.372413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:35.372413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:35.372413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:04:35.372413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:04:35.372413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:04:35.372413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:04:35.372413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:35.372413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:35.372413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:35.372413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:35.372413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:35.372413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:35.373394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:35.373394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:35.373394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:35.373394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:35.373394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:35.373394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:35.373394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:35.373394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:35.373394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:35.373394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:35.373394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:04:35.373394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:35.373394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 09:04:35.373394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:35.373394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 422/835] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:04:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:35.378456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99bc57-3a57-4eb5-b705-35da8e353f14 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:35.472367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e740ef-e1ff-4d48-98ba-412876a06131 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:35.472716 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e740ef-e1ff-4d48-98ba-412876a06131 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:35.472926 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e740ef-e1ff-4d48-98ba-412876a06131 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:35.474437 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 423/836] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:04:35.479442 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c429e3-895b-4fdc-9885-52d02609de7c None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:35.479730 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c429e3-895b-4fdc-9885-52d02609de7c None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:35.480009 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c429e3-895b-4fdc-9885-52d02609de7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:35.487750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99bc57-3a57-4eb5-b705-35da8e353f14 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:35.501130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-11c429e3-895b-4fdc-9885-52d02609de7c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:35.502901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-11c429e3-895b-4fdc-9885-52d02609de7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:35.523652 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152131f6-7c40-4318-907b-9b4a9ae68b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:35.524786 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152131f6-7c40-4318-907b-9b4a9ae68b35 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:35.525579 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152131f6-7c40-4318-907b-9b4a9ae68b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:35.526076 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152131f6-7c40-4318-907b-9b4a9ae68b35 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:35.527123 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152131f6-7c40-4318-907b-9b4a9ae68b35 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:35.527722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152131f6-7c40-4318-907b-9b4a9ae68b35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:35.528161 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:35.529857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152131f6-7c40-4318-907b-9b4a9ae68b35 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:35.537282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 414/837] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:04:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:35.541840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152131f6-7c40-4318-907b-9b4a9ae68b35 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:35.575051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:35.590747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 424/838] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 09:04:35.615697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c429e3-895b-4fdc-9885-52d02609de7c None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:35.657569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152131f6-7c40-4318-907b-9b4a9ae68b35 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:35.695464 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6470e15-e800-4140-bad6-a97568c24917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:35.696162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6470e15-e800-4140-bad6-a97568c24917 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:35.696447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6470e15-e800-4140-bad6-a97568c24917 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:35.696723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6470e15-e800-4140-bad6-a97568c24917 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:35.697435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6470e15-e800-4140-bad6-a97568c24917 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:35.697738 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6470e15-e800-4140-bad6-a97568c24917 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:35.699380 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6470e15-e800-4140-bad6-a97568c24917 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:35.711081 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 415/839] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:04:35 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:35.744493 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c429e3-895b-4fdc-9885-52d02609de7c None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:35.784483 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65393c90-bc26-4a2c-857c-2f4126e4b533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:35.785710 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65393c90-bc26-4a2c-857c-2f4126e4b533 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:35.786223 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65393c90-bc26-4a2c-857c-2f4126e4b533 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:35.786759 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65393c90-bc26-4a2c-857c-2f4126e4b533 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:35.799670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65393c90-bc26-4a2c-857c-2f4126e4b533 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:35.808062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-65393c90-bc26-4a2c-857c-2f4126e4b533 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:04:35.808062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:35.808062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:35.808062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:35.808062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:35.808062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:35.808062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:35.808062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:35.808062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:35.808062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:35.808062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:35.808062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:35.808062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:35.808062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:35.808062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:35.808062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:35.809681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:35.809681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:35.809681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:35.809681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:35.809681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:35.809681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:35.809681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:35.809681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:35.809681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:35.809681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:35.809681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:35.809681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:35.809681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:35.809681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:35.809681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:35.809681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:35.810623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:35.810623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:35.810623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:35.810623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:35.810623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:35.810623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:35.810623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:35.810623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:35.810623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:35.810623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:35.810623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:04:35.810623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:35.813599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 425/840] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:04:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:35.819434 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6470e15-e800-4140-bad6-a97568c24917 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:35.934597 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:35.941269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6470e15-e800-4140-bad6-a97568c24917 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:35.976977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2089b0-d15b-411e-aeed-583cb606c572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:35.978119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2089b0-d15b-411e-aeed-583cb606c572 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:35.978762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2089b0-d15b-411e-aeed-583cb606c572 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:35.979320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2089b0-d15b-411e-aeed-583cb606c572 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:35.980646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2089b0-d15b-411e-aeed-583cb606c572 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:35.980954 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2089b0-d15b-411e-aeed-583cb606c572 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:35.982341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2089b0-d15b-411e-aeed-583cb606c572 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:35.991137 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 416/841] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:04:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:35.996891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65393c90-bc26-4a2c-857c-2f4126e4b533 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:36.018228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:36.107329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65393c90-bc26-4a2c-857c-2f4126e4b533 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:36.142718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4bbc4-4ca7-47da-9a2d-9834c079e73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:36.143553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4bbc4-4ca7-47da-9a2d-9834c079e73b None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:36.143838 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4bbc4-4ca7-47da-9a2d-9834c079e73b None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:36.144112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4bbc4-4ca7-47da-9a2d-9834c079e73b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:36.154539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4bbc4-4ca7-47da-9a2d-9834c079e73b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:36.156595 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4bbc4-4ca7-47da-9a2d-9834c079e73b None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:36.176908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-5cf4bbc4-4ca7-47da-9a2d-9834c079e73b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:36.176908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:36.176908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:36.176908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:36.176908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:36.176908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:36.176908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:36.176908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:36.176908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:36.176908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:36.176908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:36.176908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:36.176908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:36.176908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:36.176908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:36.177930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:36.177930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:36.177930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:36.177930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:36.177930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:36.177930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:36.177930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:36.177930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:36.177930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:36.177930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:36.177930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:36.177930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:36.177930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:36.177930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:36.177930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:36.177930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:36.177930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:36.179400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:36.179400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:36.179400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:36.179400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:36.179400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 426/842] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:36.185910 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2089b0-d15b-411e-aeed-583cb606c572 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:36.203318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844306cc-3373-463b-9633-1b9550db09c3 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:36.203622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844306cc-3373-463b-9633-1b9550db09c3 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:36.203886 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844306cc-3373-463b-9633-1b9550db09c3 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:36.205510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 427/843] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:04:36.211722 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4023b17-e109-4c58-b6cf-a5f3ca87685e None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:36.211938 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4023b17-e109-4c58-b6cf-a5f3ca87685e None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:36.212139 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4023b17-e109-4c58-b6cf-a5f3ca87685e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:36.229547 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-c4023b17-e109-4c58-b6cf-a5f3ca87685e None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:36.230772 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-c4023b17-e109-4c58-b6cf-a5f3ca87685e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:36.296320 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:36.301431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2089b0-d15b-411e-aeed-583cb606c572 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:36.302037 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 428/844] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 09:04:36.315249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b47d3-8063-4fa5-bb0e-e5ca679f0570 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:36.315249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b47d3-8063-4fa5-bb0e-e5ca679f0570 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:36.315249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b47d3-8063-4fa5-bb0e-e5ca679f0570 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:36.315249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 429/845] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:36 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:36.319798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0ceae-8368-4b8e-b690-f94ea0befefb None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:36.319798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0ceae-8368-4b8e-b690-f94ea0befefb None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:36.320106 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0ceae-8368-4b8e-b690-f94ea0befefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:36.338219 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-8da0ceae-8368-4b8e-b690-f94ea0befefb None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:36.339440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-8da0ceae-8368-4b8e-b690-f94ea0befefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:36.341888 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c87c90-1c32-45f6-af7d-b5e086caf01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:36.342570 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c87c90-1c32-45f6-af7d-b5e086caf01d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:36.342861 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c87c90-1c32-45f6-af7d-b5e086caf01d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:36.343164 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c87c90-1c32-45f6-af7d-b5e086caf01d None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:36.343904 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c87c90-1c32-45f6-af7d-b5e086caf01d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:36.344214 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c87c90-1c32-45f6-af7d-b5e086caf01d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:36.345601 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c87c90-1c32-45f6-af7d-b5e086caf01d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:36.358934 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 417/846] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:36.359742 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c87c90-1c32-45f6-af7d-b5e086caf01d None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:36.380654 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:36.430938 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 430/847] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 09:04:36.430938 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0ceae-8368-4b8e-b690-f94ea0befefb None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:36.467910 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c87c90-1c32-45f6-af7d-b5e086caf01d None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:36.504462 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f5706-6fa8-4137-8897-150977dd7bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:36.505269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f5706-6fa8-4137-8897-150977dd7bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:36.505560 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f5706-6fa8-4137-8897-150977dd7bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:36.505840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f5706-6fa8-4137-8897-150977dd7bc5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:36.517028 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f5706-6fa8-4137-8897-150977dd7bc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:36.525483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-c95f5706-6fa8-4137-8897-150977dd7bc5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:36.525483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:36.525483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:36.525483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:36.525483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:36.525483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:36.525483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:36.525483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:36.525483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:36.525483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:36.525483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:36.525483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:36.525483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:36.525483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:36.525483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:36.525483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:36.526500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:36.526500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:36.526500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:36.526500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:36.526500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:36.526500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:36.526500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:36.526500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:36.526500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:36.526500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:36.526500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:36.526500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:36.526500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:36.526500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:36.526500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:36.526500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:36.528315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:36.528315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:36.528315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:36.528315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:36.528315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:36.528315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:36.528315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:36.528315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:36.528315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:36.528315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:36.528315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:04:36.528315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:36.528315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 418/848] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:04:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:36.528315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f5706-6fa8-4137-8897-150977dd7bc5 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:36.536215 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0ceae-8368-4b8e-b690-f94ea0befefb None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:36.563598 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:36.571057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22592fb-9281-40ba-8df9-0caf67bad54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:36.571975 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22592fb-9281-40ba-8df9-0caf67bad54d None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:36.572265 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22592fb-9281-40ba-8df9-0caf67bad54d None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:36.572545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22592fb-9281-40ba-8df9-0caf67bad54d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:36.579830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22592fb-9281-40ba-8df9-0caf67bad54d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:36.581449 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22592fb-9281-40ba-8df9-0caf67bad54d None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:36.596537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-a22592fb-9281-40ba-8df9-0caf67bad54d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 09:04:36.596537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:36.596537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:36.596537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:36.596537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:36.596537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:36.596537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:36.596537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:36.596537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:36.596537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:36.596537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:36.596537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:36.596537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:36.596537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:36.596537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:36.596537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:36.597523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:36.597523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:36.597523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:36.597523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:36.597523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:36.597523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:36.597523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:36.597523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:36.597523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:36.597523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:36.597523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:36.597523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:36.597523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:36.597523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:36.597523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:36.597523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:36.597523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:36.597523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:36.599191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:36.599191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:36.599191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:36.599191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:36.599191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:36.599191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:36.599191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:36.599191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:36.599191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 09:04:36.599191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:36.599191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 431/849] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:04:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:36.599191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22592fb-9281-40ba-8df9-0caf67bad54d None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:36.621803 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:36.644786 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f5706-6fa8-4137-8897-150977dd7bc5 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:36.680734 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639e4c7-2d56-46fa-aeda-0474b07f07ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:36.681961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639e4c7-2d56-46fa-aeda-0474b07f07ca None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:36.682253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639e4c7-2d56-46fa-aeda-0474b07f07ca None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:36.682534 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639e4c7-2d56-46fa-aeda-0474b07f07ca None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:36.684111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f639e4c7-2d56-46fa-aeda-0474b07f07ca None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:36.686088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f639e4c7-2d56-46fa-aeda-0474b07f07ca None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:36.702376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-f639e4c7-2d56-46fa-aeda-0474b07f07ca None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 09:04:36.702376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:36.702376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:04:36.702376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:04:36.702376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:04:36.702376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:04:36.702376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:04:36.702376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:36.702376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 09:04:36.702376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:36.702376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:36.702376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:36.702376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:36.702376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:36.702376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:36.702376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:36.703405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:36.703405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:36.703405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:36.703405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:36.703405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:36.703405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:36.703405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:36.703405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:04:36.703405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:04:36.703405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:04:36.703405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:04:36.703405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:36.703405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:36.703405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:36.703405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:36.703405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:36.704290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:36.704290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:36.704290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:36.704290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:36.704290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:36.704290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:36.704290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:36.704290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:36.704290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:36.704290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:36.704290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:36.704290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:04:36.704290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:36.704290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 09:04:36.704290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:36.705926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 419/850] 10.222.0.52 () {56 vars in 1132 bytes} [Thu Jun 23 09:04:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:36.706742 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639e4c7-2d56-46fa-aeda-0474b07f07ca None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:36.711301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22592fb-9281-40ba-8df9-0caf67bad54d None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:36.745969 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029293fa-4cb8-4ef5-8196-f433c2935b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:36.746843 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029293fa-4cb8-4ef5-8196-f433c2935b70 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:36.747350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029293fa-4cb8-4ef5-8196-f433c2935b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:36.747804 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029293fa-4cb8-4ef5-8196-f433c2935b70 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:36.748994 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029293fa-4cb8-4ef5-8196-f433c2935b70 None admin] RBAC: Adding query filter params (name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:36.749486 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029293fa-4cb8-4ef5-8196-f433c2935b70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:36.751494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029293fa-4cb8-4ef5-8196-f433c2935b70 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:36.760103 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 432/851] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jun 23 09:04:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:36.760902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029293fa-4cb8-4ef5-8196-f433c2935b70 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:36.814567 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:36.823133 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639e4c7-2d56-46fa-aeda-0474b07f07ca None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:36.858160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481497a-31d8-4392-8d1b-e85eef5789a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:36.858834 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481497a-31d8-4392-8d1b-e85eef5789a3 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:36.859224 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481497a-31d8-4392-8d1b-e85eef5789a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:36.859500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481497a-31d8-4392-8d1b-e85eef5789a3 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:36.860344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481497a-31d8-4392-8d1b-e85eef5789a3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:36.860656 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481497a-31d8-4392-8d1b-e85eef5789a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:36.862090 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481497a-31d8-4392-8d1b-e85eef5789a3 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:36.870765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029293fa-4cb8-4ef5-8196-f433c2935b70 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:36.872347 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:36.885205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 420/852] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jun 23 09:04:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:36.885205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481497a-31d8-4392-8d1b-e85eef5789a3 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:36.912121 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa4c60-aefa-4660-bfbf-f359e53391ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:36.912802 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa4c60-aefa-4660-bfbf-f359e53391ca None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:36.913082 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa4c60-aefa-4660-bfbf-f359e53391ca None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:36.913365 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa4c60-aefa-4660-bfbf-f359e53391ca None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:36.914078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa4c60-aefa-4660-bfbf-f359e53391ca None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:36.914383 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa4c60-aefa-4660-bfbf-f359e53391ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:36.915779 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa4c60-aefa-4660-bfbf-f359e53391ca None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:36.921514 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 433/853] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jun 23 09:04:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:36.922395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa4c60-aefa-4660-bfbf-f359e53391ca None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:36.998257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481497a-31d8-4392-8d1b-e85eef5789a3 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:37.032816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa4c60-aefa-4660-bfbf-f359e53391ca None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:37.033760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7085c50-9112-4bd8-880d-eaa44445c935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:37.034621 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7085c50-9112-4bd8-880d-eaa44445c935 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:37.035006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7085c50-9112-4bd8-880d-eaa44445c935 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:37.035344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7085c50-9112-4bd8-880d-eaa44445c935 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:37.046235 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7085c50-9112-4bd8-880d-eaa44445c935 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:37.047709 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7085c50-9112-4bd8-880d-eaa44445c935 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:37.048220 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:37.065365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-d7085c50-9112-4bd8-880d-eaa44445c935 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:37.065365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:37.065365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:37.065365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:37.065365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:37.065365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:37.065365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:37.065365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:37.065365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:37.065365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:37.065365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:37.065365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:37.065365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:37.065365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:37.065365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:37.066366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:37.066366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:37.066366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:37.066366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:37.066366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:37.066366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:37.066366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:37.066366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:37.066366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:37.066366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:37.066366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:37.066366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:37.066366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:37.066366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:37.066366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:37.066366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:37.066366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:37.067822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:37.067822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:37.067822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:37.067822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:37.067822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 421/854] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:37.067822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7085c50-9112-4bd8-880d-eaa44445c935 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:37.073078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47479eb-9a95-4d8e-b8df-705a72da3337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:37.073739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47479eb-9a95-4d8e-b8df-705a72da3337 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:37.074605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47479eb-9a95-4d8e-b8df-705a72da3337 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:37.074890 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47479eb-9a95-4d8e-b8df-705a72da3337 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:37.126555 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:37.147755 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47479eb-9a95-4d8e-b8df-705a72da3337 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:37.149254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47479eb-9a95-4d8e-b8df-705a72da3337 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:37.184628 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7085c50-9112-4bd8-880d-eaa44445c935 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:37.221619 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd8ca14-c069-49f1-8cdd-0fff0b550c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:37.222259 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8ca14-c069-49f1-8cdd-0fff0b550c2d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:37.222553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8ca14-c069-49f1-8cdd-0fff0b550c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:37.222835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8ca14-c069-49f1-8cdd-0fff0b550c2d None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:37.223571 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd8ca14-c069-49f1-8cdd-0fff0b550c2d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:37.223879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd8ca14-c069-49f1-8cdd-0fff0b550c2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:37.225251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd8ca14-c069-49f1-8cdd-0fff0b550c2d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:37.229640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 422/855] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jun 23 09:04:37 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:37.230438 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd8ca14-c069-49f1-8cdd-0fff0b550c2d None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:37.270747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 434/856] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:04:36 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 09:04:37.276076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47479eb-9a95-4d8e-b8df-705a72da3337 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:37.294261 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:37.389811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd8ca14-c069-49f1-8cdd-0fff0b550c2d None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:37.391961 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47479eb-9a95-4d8e-b8df-705a72da3337 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:37.407034 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:37.432531 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411e5f0-a8f2-4ba9-8b26-7000500fbb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:37.433207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411e5f0-a8f2-4ba9-8b26-7000500fbb05 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:37.433538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411e5f0-a8f2-4ba9-8b26-7000500fbb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:37.433820 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411e5f0-a8f2-4ba9-8b26-7000500fbb05 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:37.434096 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617c23c-cf6c-4d4e-85ba-13dab0650d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:37.434552 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411e5f0-a8f2-4ba9-8b26-7000500fbb05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:37.434863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411e5f0-a8f2-4ba9-8b26-7000500fbb05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:37.435000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617c23c-cf6c-4d4e-85ba-13dab0650d76 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:37.435937 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617c23c-cf6c-4d4e-85ba-13dab0650d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:37.436217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617c23c-cf6c-4d4e-85ba-13dab0650d76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:37.436333 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411e5f0-a8f2-4ba9-8b26-7000500fbb05 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:37.447732 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617c23c-cf6c-4d4e-85ba-13dab0650d76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:37.449293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617c23c-cf6c-4d4e-85ba-13dab0650d76 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:37.449418 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 423/857] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:37.454770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411e5f0-a8f2-4ba9-8b26-7000500fbb05 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:37.476778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-6617c23c-cf6c-4d4e-85ba-13dab0650d76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:37.476778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:37.476778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:37.476778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:37.476778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:37.476778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:37.476778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:37.476778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:37.476778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:37.476778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:37.476778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:37.476778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:37.476778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:37.476778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:37.476778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:37.477789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:37.477789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:37.477789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:37.477789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:37.477789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:37.477789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:37.477789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:37.477789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:37.477789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:37.477789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:37.477789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:37.477789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:37.477789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:37.477789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:37.477789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:37.477789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:37.477789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:37.478751 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:37.478751 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:37.478751 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:37.478751 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:37.478751 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 435/858] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:37.484865 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617c23c-cf6c-4d4e-85ba-13dab0650d76 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:37.566672 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411e5f0-a8f2-4ba9-8b26-7000500fbb05 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:37.586409 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:37.602649 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85fc6d-5dcf-4dc2-8b5b-3ac59b257245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:37.603642 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85fc6d-5dcf-4dc2-8b5b-3ac59b257245 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:37.603986 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85fc6d-5dcf-4dc2-8b5b-3ac59b257245 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:37.604324 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85fc6d-5dcf-4dc2-8b5b-3ac59b257245 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:37.613556 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef85fc6d-5dcf-4dc2-8b5b-3ac59b257245 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:37.615202 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef85fc6d-5dcf-4dc2-8b5b-3ac59b257245 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:37.619270 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617c23c-cf6c-4d4e-85ba-13dab0650d76 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:37.631431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-ef85fc6d-5dcf-4dc2-8b5b-3ac59b257245 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 09:04:37.631431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:37.631431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:37.631431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:37.631431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:37.631431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:37.631431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:37.631431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:37.631431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:37.631431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:37.631431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:37.631431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:37.631431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:37.631431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:37.631431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:37.631431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:37.632676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:37.632676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:37.632676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:37.632676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:37.632676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:37.632676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:37.632676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:37.632676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:37.632676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:37.632676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:37.632676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:37.632676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:37.632676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:37.632676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:37.632676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:37.632676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:37.632676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:37.632676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:37.633932 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:37.633932 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:37.633932 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:37.633932 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:37.633932 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:37.633932 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:37.633932 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:37.633932 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:37.633932 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 09:04:37.633932 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:37.640519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 424/859] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jun 23 09:04:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:37.641517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85fc6d-5dcf-4dc2-8b5b-3ac59b257245 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:37.663925 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4a957-6f7b-4de2-942c-286105bef86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:37.665395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4a957-6f7b-4de2-942c-286105bef86e None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:37.666093 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4a957-6f7b-4de2-942c-286105bef86e None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:37.666541 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4a957-6f7b-4de2-942c-286105bef86e None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:37.668263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4a957-6f7b-4de2-942c-286105bef86e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:37.668755 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4a957-6f7b-4de2-942c-286105bef86e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:37.670963 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4a957-6f7b-4de2-942c-286105bef86e None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:37.678739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:37.687272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 436/860] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:37.692793 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4a957-6f7b-4de2-942c-286105bef86e None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:37.755666 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85fc6d-5dcf-4dc2-8b5b-3ac59b257245 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:37.800862 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7bc499-29f1-47a1-94dd-2cc085cec1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:37.802303 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bc499-29f1-47a1-94dd-2cc085cec1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:37.802840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bc499-29f1-47a1-94dd-2cc085cec1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:37.803358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bc499-29f1-47a1-94dd-2cc085cec1d0 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:37.804426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7bc499-29f1-47a1-94dd-2cc085cec1d0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:37.805035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7bc499-29f1-47a1-94dd-2cc085cec1d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:37.806822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7bc499-29f1-47a1-94dd-2cc085cec1d0 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:37.825200 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4a957-6f7b-4de2-942c-286105bef86e None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:37.829062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 425/861] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jun 23 09:04:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:37.836582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7bc499-29f1-47a1-94dd-2cc085cec1d0 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:37.863019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:37.872016 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c715d-f81b-40bb-a601-186f1ec5be8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:37.875640 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c715d-f81b-40bb-a601-186f1ec5be8a None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:37.875640 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c715d-f81b-40bb-a601-186f1ec5be8a None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:37.875640 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c715d-f81b-40bb-a601-186f1ec5be8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:37.881271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c715d-f81b-40bb-a601-186f1ec5be8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:37.884020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c715d-f81b-40bb-a601-186f1ec5be8a None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:37.904677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-fc8c715d-f81b-40bb-a601-186f1ec5be8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:37.904677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:37.904677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:37.904677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:37.904677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:37.904677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:37.904677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:37.904677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:37.904677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:37.904677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:37.904677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:37.904677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:37.904677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:37.904677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:37.904677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:37.905740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:37.905740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:37.905740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:37.905740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:37.905740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:37.905740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:37.905740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:37.905740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:37.905740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:37.905740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:37.905740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:37.905740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:37.905740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:37.905740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:37.905740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:37.905740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:37.905740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:37.906698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:37.906698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:37.906698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:37.906698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:37.906698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:37.906698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:37.906698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:37.906698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:37.907785 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 437/862] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:37.908660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbad96a-4c69-4a76-9372-0cd57d715a59 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:37.908955 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbad96a-4c69-4a76-9372-0cd57d715a59 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:37.909240 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbad96a-4c69-4a76-9372-0cd57d715a59 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:37.910645 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 438/863] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:37.915395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbad96a-4c69-4a76-9372-0cd57d715a59 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:37.950895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:37.959000 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7bc499-29f1-47a1-94dd-2cc085cec1d0 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:37.995294 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea189816-30f6-42a1-abe8-872c39313c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:37.996000 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea189816-30f6-42a1-abe8-872c39313c99 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:37.996287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea189816-30f6-42a1-abe8-872c39313c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:37.996559 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea189816-30f6-42a1-abe8-872c39313c99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:38.025729 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbad96a-4c69-4a76-9372-0cd57d715a59 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:38.065132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92289cc-9cad-4b59-a3e6-98a89f317229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:38.066013 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92289cc-9cad-4b59-a3e6-98a89f317229 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:38.066416 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92289cc-9cad-4b59-a3e6-98a89f317229 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:38.066795 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92289cc-9cad-4b59-a3e6-98a89f317229 None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:38.067714 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92289cc-9cad-4b59-a3e6-98a89f317229 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:38.068132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92289cc-9cad-4b59-a3e6-98a89f317229 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:38.069650 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92289cc-9cad-4b59-a3e6-98a89f317229 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:38.071743 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea189816-30f6-42a1-abe8-872c39313c99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:38.072747 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea189816-30f6-42a1-abe8-872c39313c99 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:38.084788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 439/864] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:38.084788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46303082-ea73-4d9b-b217-2e1ddf28fb35 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:38.084788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46303082-ea73-4d9b-b217-2e1ddf28fb35 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:38.084788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46303082-ea73-4d9b-b217-2e1ddf28fb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:38.102333 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-46303082-ea73-4d9b-b217-2e1ddf28fb35 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:38.103575 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-46303082-ea73-4d9b-b217-2e1ddf28fb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:38.148721 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:38.172403 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:38.186498 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 426/865] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:04:37 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 09:04:38.186498 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea189816-30f6-42a1-abe8-872c39313c99 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:38.186789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 440/866] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:04:38.187701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46303082-ea73-4d9b-b217-2e1ddf28fb35 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:38.305145 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea189816-30f6-42a1-abe8-872c39313c99 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:38.314779 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46303082-ea73-4d9b-b217-2e1ddf28fb35 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:38.342591 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e9b5f-a617-4aca-881d-5ef6e370a256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:38.343203 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e9b5f-a617-4aca-881d-5ef6e370a256 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:38.343500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e9b5f-a617-4aca-881d-5ef6e370a256 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:38.343775 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e9b5f-a617-4aca-881d-5ef6e370a256 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:38.344610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e9b5f-a617-4aca-881d-5ef6e370a256 None admin] RBAC: Adding query filter params (role.id=9cd080b50c6f4ee8a54bf9ef1655bd83, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=5b1331b4f0924bb8b137a22a33c9d130) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:38.344928 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e9b5f-a617-4aca-881d-5ef6e370a256 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:38.346371 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e9b5f-a617-4aca-881d-5ef6e370a256 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:38.352603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 427/867] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:04:38 2022] GET /identity/v3/role_assignments?user.id=5b1331b4f0924bb8b137a22a33c9d130&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:38.353224 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e9b5f-a617-4aca-881d-5ef6e370a256 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:38.363649 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eeea96-10c7-4251-afc2-fa01d8c5b36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:38.364902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eeea96-10c7-4251-afc2-fa01d8c5b36e None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:38.366011 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eeea96-10c7-4251-afc2-fa01d8c5b36e None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:38.366530 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eeea96-10c7-4251-afc2-fa01d8c5b36e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:38.380340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eeea96-10c7-4251-afc2-fa01d8c5b36e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:38.382203 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eeea96-10c7-4251-afc2-fa01d8c5b36e None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:38.403369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-19eeea96-10c7-4251-afc2-fa01d8c5b36e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:38.403369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:38.403369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:38.403369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:38.403369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:38.403369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:38.403369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:38.403369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:38.403369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:38.403369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:38.403369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:38.403369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:38.403369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:38.403369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:38.403369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:38.404417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:38.404417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:38.404417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:38.404417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:38.404417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:38.404417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:38.404417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:38.404417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:38.404417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:38.404417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:38.404417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:38.404417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:38.404417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:38.404417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:38.404417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:38.404417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:38.404417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:38.405413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:38.405413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:38.405413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:38.405413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:38.407898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 441/868] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:38.413541 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eeea96-10c7-4251-afc2-fa01d8c5b36e None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:38.434936 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:38.454352 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:38.500168 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e9b5f-a617-4aca-881d-5ef6e370a256 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:38.542410 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eeea96-10c7-4251-afc2-fa01d8c5b36e None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:38.546241 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e003f6-1ba0-4e41-8af4-77f8ee6c9374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:38.547226 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e003f6-1ba0-4e41-8af4-77f8ee6c9374 None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:38.547756 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e003f6-1ba0-4e41-8af4-77f8ee6c9374 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:38.548259 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e003f6-1ba0-4e41-8af4-77f8ee6c9374 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:38.549265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e003f6-1ba0-4e41-8af4-77f8ee6c9374 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:38.551067 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e003f6-1ba0-4e41-8af4-77f8ee6c9374 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:38.580908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b685fee-2cd8-480a-83f5-dd6cb24082e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:38.581637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b685fee-2cd8-480a-83f5-dd6cb24082e8 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:38.581925 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b685fee-2cd8-480a-83f5-dd6cb24082e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:38.582207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b685fee-2cd8-480a-83f5-dd6cb24082e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:38.583009 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b685fee-2cd8-480a-83f5-dd6cb24082e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:38.583327 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b685fee-2cd8-480a-83f5-dd6cb24082e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:38.584962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b685fee-2cd8-480a-83f5-dd6cb24082e8 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:38.597308 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 442/869] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:38.602509 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b685fee-2cd8-480a-83f5-dd6cb24082e8 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:38.619731 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 428/870] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jun 23 09:04:38 2022] POST /identity/v3/services => generated 251 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:04:38.713367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:38.727724 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b685fee-2cd8-480a-83f5-dd6cb24082e8 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:38.774449 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c75e6-36f3-4308-8f16-d163b1499c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:38.775298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c75e6-36f3-4308-8f16-d163b1499c95 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:38.775590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c75e6-36f3-4308-8f16-d163b1499c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:38.775876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c75e6-36f3-4308-8f16-d163b1499c95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:38.783993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c75e6-36f3-4308-8f16-d163b1499c95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:38.785457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c75e6-36f3-4308-8f16-d163b1499c95 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:38.800580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-ee3c75e6-36f3-4308-8f16-d163b1499c95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:38.800580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:38.800580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:38.800580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:38.800580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:38.800580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:38.800580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:38.800580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:38.800580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:38.800580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:38.800580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:38.800580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:38.800580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:38.800580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:38.800580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:38.801542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:38.801542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:38.801542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:38.801542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:38.801542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:38.801542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:38.801542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:38.801542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:38.801542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:38.801542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:38.801542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:38.801542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:38.801542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:38.801542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:38.801542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:38.801542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:38.801542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:38.802558 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:38.802558 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:38.802558 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:38.802558 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:38.802558 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:38.802558 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:38.802558 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:38.802558 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:38.802558 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 443/871] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:38.809386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e003f6-1ba0-4e41-8af4-77f8ee6c9374 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:38.919275 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e003f6-1ba0-4e41-8af4-77f8ee6c9374 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:38.940532 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:38.955302 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c133e-1b5d-4974-a7fd-b4acec35cf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:38.965668 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c133e-1b5d-4974-a7fd-b4acec35cf3e None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:38.965668 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c133e-1b5d-4974-a7fd-b4acec35cf3e None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:38.965668 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c133e-1b5d-4974-a7fd-b4acec35cf3e None admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:38.965668 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c133e-1b5d-4974-a7fd-b4acec35cf3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:38.965668 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c133e-1b5d-4974-a7fd-b4acec35cf3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:38.965668 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c133e-1b5d-4974-a7fd-b4acec35cf3e None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:38.972958 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 429/872] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:38.978738 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c75e6-36f3-4308-8f16-d163b1499c95 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:39.093481 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c75e6-36f3-4308-8f16-d163b1499c95 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:39.129072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f016181-b7a0-4378-b5e2-218f354c1c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:39.129746 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f016181-b7a0-4378-b5e2-218f354c1c74 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:39.130058 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f016181-b7a0-4378-b5e2-218f354c1c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:39.130361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f016181-b7a0-4378-b5e2-218f354c1c74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:39.131245 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f016181-b7a0-4378-b5e2-218f354c1c74 None admin] RBAC: Adding query filter params (role.id=edf696ae763147cab68434524d1eff9c, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=cfbaba84402d4c31bafcd12133d74c03) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:39.131566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f016181-b7a0-4378-b5e2-218f354c1c74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:39.133010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f016181-b7a0-4378-b5e2-218f354c1c74 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:39.140108 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 444/873] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:04:38 2022] GET /identity/v3/role_assignments?user.id=cfbaba84402d4c31bafcd12133d74c03&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=edf696ae763147cab68434524d1eff9c => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:39.376928 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf57375-ba6b-4e4d-a8ba-9701579cf508 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:39.377477 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf57375-ba6b-4e4d-a8ba-9701579cf508 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:39.377477 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf57375-ba6b-4e4d-a8ba-9701579cf508 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:39.379158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 430/874] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:39.384410 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c43992-1430-489d-80f4-c90a6ec27905 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:39.384837 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c43992-1430-489d-80f4-c90a6ec27905 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:39.385002 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c43992-1430-489d-80f4-c90a6ec27905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:39.403253 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-85c43992-1430-489d-80f4-c90a6ec27905 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:39.404444 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-85c43992-1430-489d-80f4-c90a6ec27905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:39.406426 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:39.479148 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 445/875] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:39 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:04:39.497934 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf57375-ba6b-4e4d-a8ba-9701579cf508 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:39.608379 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf57375-ba6b-4e4d-a8ba-9701579cf508 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:39.644019 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48691799-6829-48dc-a04e-09266a30623a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:39.644609 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48691799-6829-48dc-a04e-09266a30623a None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:39.644899 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48691799-6829-48dc-a04e-09266a30623a None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:39.645180 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48691799-6829-48dc-a04e-09266a30623a None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:39.645919 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48691799-6829-48dc-a04e-09266a30623a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:39.647346 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48691799-6829-48dc-a04e-09266a30623a None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:39.745331 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 431/876] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jun 23 09:04:39 2022] POST /identity/v3/services => generated 269 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:04:40.607723 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622dbe9-807c-4e4c-9385-0e15d20fa749 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:40.608329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622dbe9-807c-4e4c-9385-0e15d20fa749 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:40.608329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622dbe9-807c-4e4c-9385-0e15d20fa749 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:40.609799 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 446/877] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:40.615171 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f8f2e-a6b2-4ce5-98cc-b7bbdeda0db1 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:40.615959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f8f2e-a6b2-4ce5-98cc-b7bbdeda0db1 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:40.616478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f8f2e-a6b2-4ce5-98cc-b7bbdeda0db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:40.635608 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-db9f8f2e-a6b2-4ce5-98cc-b7bbdeda0db1 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:40.637628 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-db9f8f2e-a6b2-4ce5-98cc-b7bbdeda0db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:40.669081 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:40.717327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 432/878] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:04:40.739581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622dbe9-807c-4e4c-9385-0e15d20fa749 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:40.844410 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622dbe9-807c-4e4c-9385-0e15d20fa749 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:40.868811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7395e-66e3-4591-862b-9a871e745914 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:40.869007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7395e-66e3-4591-862b-9a871e745914 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:40.869292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7395e-66e3-4591-862b-9a871e745914 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:40.874651 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 433/879] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:04:40.876069 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39fa09-c98d-4552-8b53-993af53e59d3 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:40.876631 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39fa09-c98d-4552-8b53-993af53e59d3 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:40.876834 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39fa09-c98d-4552-8b53-993af53e59d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:40.881564 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a27f5-5d68-4ff0-ba4c-09fdc43752b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:40.882672 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a27f5-5d68-4ff0-ba4c-09fdc43752b4 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:40.883256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a27f5-5d68-4ff0-ba4c-09fdc43752b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:40.883667 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a27f5-5d68-4ff0-ba4c-09fdc43752b4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:40.884451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a27f5-5d68-4ff0-ba4c-09fdc43752b4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:40.885976 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a27f5-5d68-4ff0-ba4c-09fdc43752b4 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:40.894831 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-ed39fa09-c98d-4552-8b53-993af53e59d3 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:40.896202 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ed39fa09-c98d-4552-8b53-993af53e59d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:40.899810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-292a27f5-5d68-4ff0-ba4c-09fdc43752b4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 09:04:40.899810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:40.899810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:04:40.899810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:04:40.899810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:04:40.899810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:04:40.899810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:04:40.899810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:40.899810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 09:04:40.899810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:40.899810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:40.899810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:40.899810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:40.899810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:40.899810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:40.899810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:40.899810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:40.901050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:40.901050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:40.901050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:40.901050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:40.901050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:40.901050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:40.901050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:04:40.901050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:04:40.901050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:04:40.901050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:04:40.901050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:40.901050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:40.901050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:40.901050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:40.901050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:40.901050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:40.902681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:40.902681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:40.902681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:40.902681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:40.902681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:40.902681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:40.902681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:40.902681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:40.902681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:40.902681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:40.902681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:04:40.902681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:40.902681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 09:04:40.902681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:40.905485 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 447/880] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jun 23 09:04:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:40.914517 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a27f5-5d68-4ff0-ba4c-09fdc43752b4 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:40.951289 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:40.977979 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 434/881] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 09:04:41.001974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39fa09-c98d-4552-8b53-993af53e59d3 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:41.033622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a27f5-5d68-4ff0-ba4c-09fdc43752b4 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:41.077641 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6192cc-bbfe-4a07-b45b-85627be8dcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:41.079382 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6192cc-bbfe-4a07-b45b-85627be8dcb3 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:41.079886 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6192cc-bbfe-4a07-b45b-85627be8dcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:41.080632 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6192cc-bbfe-4a07-b45b-85627be8dcb3 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:41.082156 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6192cc-bbfe-4a07-b45b-85627be8dcb3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:41.082762 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6192cc-bbfe-4a07-b45b-85627be8dcb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:41.085354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6192cc-bbfe-4a07-b45b-85627be8dcb3 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:41.092475 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 448/882] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jun 23 09:04:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:41.097932 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6192cc-bbfe-4a07-b45b-85627be8dcb3 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:41.130192 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:41.141719 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39fa09-c98d-4552-8b53-993af53e59d3 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:41.193762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c82c9f-86ea-4333-b840-791863f03549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:41.194625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c82c9f-86ea-4333-b840-791863f03549 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:41.194924 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c82c9f-86ea-4333-b840-791863f03549 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:41.195267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c82c9f-86ea-4333-b840-791863f03549 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:41.195923 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c82c9f-86ea-4333-b840-791863f03549 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:41.197355 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c82c9f-86ea-4333-b840-791863f03549 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:41.199748 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:41.210625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-a6c82c9f-86ea-4333-b840-791863f03549 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 09:04:41.210625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:41.210625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:04:41.210625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:04:41.210625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:04:41.210625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:04:41.210625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:04:41.210625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:41.210625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 09:04:41.210625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:41.210625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:41.210625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:41.210625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:41.210625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:41.210625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:41.210625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:41.210625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:41.216118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:41.216118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:41.216118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:41.216118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:41.216118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:41.216118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:41.216118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:04:41.216118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:04:41.216118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:04:41.216118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:04:41.216118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:41.216118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:41.216118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:41.216118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:41.216118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:41.216118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:41.217016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:41.217016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:41.217016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:41.217016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:41.217016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:41.217016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:41.217016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:41.217016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:41.217016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:41.217016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:41.217016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:04:41.217016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:41.217016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 09:04:41.217016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:41.217016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 435/883] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:04:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:41.219092 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c82c9f-86ea-4333-b840-791863f03549 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:41.244709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6192cc-bbfe-4a07-b45b-85627be8dcb3 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:41.281169 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049cb51-0aef-411f-8c66-09d3fcf1ef9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:41.282151 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049cb51-0aef-411f-8c66-09d3fcf1ef9f None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:41.282632 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049cb51-0aef-411f-8c66-09d3fcf1ef9f None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:41.283147 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049cb51-0aef-411f-8c66-09d3fcf1ef9f None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:41.284103 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2049cb51-0aef-411f-8c66-09d3fcf1ef9f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:41.284611 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2049cb51-0aef-411f-8c66-09d3fcf1ef9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:41.286185 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2049cb51-0aef-411f-8c66-09d3fcf1ef9f None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:41.297023 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 449/884] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jun 23 09:04:41 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:41.298589 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552d74e-c201-48f5-b3b9-ed9ee40c1172 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:41.298885 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552d74e-c201-48f5-b3b9-ed9ee40c1172 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:41.299218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552d74e-c201-48f5-b3b9-ed9ee40c1172 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:41.300695 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 450/885] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:41.306596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152cb05-5311-4f5f-acde-2079ea2e3972 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:41.306903 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152cb05-5311-4f5f-acde-2079ea2e3972 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:41.307258 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152cb05-5311-4f5f-acde-2079ea2e3972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:41.329690 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-e152cb05-5311-4f5f-acde-2079ea2e3972 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:41.332422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e152cb05-5311-4f5f-acde-2079ea2e3972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:41.334538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c82c9f-86ea-4333-b840-791863f03549 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:41.373612 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846b778-be18-4213-a7c8-392974b552e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:41.374675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846b778-be18-4213-a7c8-392974b552e6 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:41.375058 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846b778-be18-4213-a7c8-392974b552e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:41.375411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846b778-be18-4213-a7c8-392974b552e6 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:41.376245 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3846b778-be18-4213-a7c8-392974b552e6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:41.376626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3846b778-be18-4213-a7c8-392974b552e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:41.378137 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3846b778-be18-4213-a7c8-392974b552e6 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:41.388882 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 436/886] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:04:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:41.390068 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846b778-be18-4213-a7c8-392974b552e6 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:41.417153 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:41.431209 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 451/887] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 09:04:41.446424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152cb05-5311-4f5f-acde-2079ea2e3972 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:41.507582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846b778-be18-4213-a7c8-392974b552e6 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:41.516302 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:41.553137 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800f08c-bdd7-4457-a929-54a9baf50674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:41.554498 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800f08c-bdd7-4457-a929-54a9baf50674 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:41.555184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800f08c-bdd7-4457-a929-54a9baf50674 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:41.555966 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800f08c-bdd7-4457-a929-54a9baf50674 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:41.557207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6800f08c-bdd7-4457-a929-54a9baf50674 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:41.557729 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6800f08c-bdd7-4457-a929-54a9baf50674 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:41.558613 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152cb05-5311-4f5f-acde-2079ea2e3972 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:41.559697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6800f08c-bdd7-4457-a929-54a9baf50674 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:41.567303 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 437/888] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:04:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:41.572379 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800f08c-bdd7-4457-a929-54a9baf50674 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:41.602615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591773a-c5ec-485f-b0ba-0075b9b2f939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:41.603518 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591773a-c5ec-485f-b0ba-0075b9b2f939 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:41.604139 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591773a-c5ec-485f-b0ba-0075b9b2f939 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:41.604139 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591773a-c5ec-485f-b0ba-0075b9b2f939 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:41.615839 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591773a-c5ec-485f-b0ba-0075b9b2f939 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:41.624316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-b591773a-c5ec-485f-b0ba-0075b9b2f939 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 09:04:41.624316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:41.624316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:41.624316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:41.624316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:41.624316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:41.624316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:41.624316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:41.624316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:41.624316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:41.624316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:41.624316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:41.624316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:41.624316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:41.624316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:41.624316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:41.625337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:41.625337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:41.625337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:41.625337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:41.625337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:41.625337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:41.625337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:41.625337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:41.625337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:41.625337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:41.625337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:41.625337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:41.625337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:41.625337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:41.625337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:41.625337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:41.626277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:41.626277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:41.626277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:41.626277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:41.626277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:41.626277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:41.626277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:41.626277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:41.626277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:41.626277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:41.626277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 09:04:41.626277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:41.630672 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 452/889] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jun 23 09:04:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:41.631750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591773a-c5ec-485f-b0ba-0075b9b2f939 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:41.693502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:41.718053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800f08c-bdd7-4457-a929-54a9baf50674 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:41.742426 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591773a-c5ec-485f-b0ba-0075b9b2f939 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:41.766706 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ab5de-4cdd-4485-9fc5-21d4f35330ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:41.767327 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:41.767444 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ab5de-4cdd-4485-9fc5-21d4f35330ea None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:41.767722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ab5de-4cdd-4485-9fc5-21d4f35330ea None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:41.768009 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ab5de-4cdd-4485-9fc5-21d4f35330ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:41.769165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ab5de-4cdd-4485-9fc5-21d4f35330ea None admin] RBAC: Adding query filter params (interface=public, service_id=89d2d3593eeb443e8dcfb406f2ac57e6, region_id=RegionOne) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:41.769165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ab5de-4cdd-4485-9fc5-21d4f35330ea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:41.770577 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ab5de-4cdd-4485-9fc5-21d4f35330ea None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:41.780660 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 438/890] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jun 23 09:04:41 2022] GET /identity/v3/endpoints?service_id=89d2d3593eeb443e8dcfb406f2ac57e6&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:41.782843 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ab5de-4cdd-4485-9fc5-21d4f35330ea None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:41.786115 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc88df-8c10-4c54-bc5b-654bd920ae73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:41.787899 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc88df-8c10-4c54-bc5b-654bd920ae73 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:41.788406 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc88df-8c10-4c54-bc5b-654bd920ae73 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:41.788922 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc88df-8c10-4c54-bc5b-654bd920ae73 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:41.789886 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc88df-8c10-4c54-bc5b-654bd920ae73 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:41.790417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc88df-8c10-4c54-bc5b-654bd920ae73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:41.792022 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc88df-8c10-4c54-bc5b-654bd920ae73 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:41.805413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 453/891] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jun 23 09:04:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:41.811740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc88df-8c10-4c54-bc5b-654bd920ae73 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:41.907471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ab5de-4cdd-4485-9fc5-21d4f35330ea None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:41.932675 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc88df-8c10-4c54-bc5b-654bd920ae73 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:41.953208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66189fa9-5f4a-4ca6-9e4b-d9e111a007ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:41.954129 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66189fa9-5f4a-4ca6-9e4b-d9e111a007ae None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:41.954597 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66189fa9-5f4a-4ca6-9e4b-d9e111a007ae None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:41.955058 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66189fa9-5f4a-4ca6-9e4b-d9e111a007ae None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:41.955926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66189fa9-5f4a-4ca6-9e4b-d9e111a007ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:41.957560 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66189fa9-5f4a-4ca6-9e4b-d9e111a007ae None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:41.964195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 439/892] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jun 23 09:04:41 2022] GET /identity/v3/services => generated 1268 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:04:41.966240 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b1241-da74-42bb-b7e3-a8cc7accbd9b None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:41.966535 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b1241-da74-42bb-b7e3-a8cc7accbd9b None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:41.966843 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b1241-da74-42bb-b7e3-a8cc7accbd9b None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:41.970961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 440/893] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:41.974226 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a87f9-6547-4ac7-a5ba-5a3512598551 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:41.974814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a87f9-6547-4ac7-a5ba-5a3512598551 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:41.975339 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a87f9-6547-4ac7-a5ba-5a3512598551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:41.977119 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f607694-a399-48c3-8c5f-fb0a3c2d7da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:41.977997 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f607694-a399-48c3-8c5f-fb0a3c2d7da6 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:41.978355 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f607694-a399-48c3-8c5f-fb0a3c2d7da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:41.978748 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f607694-a399-48c3-8c5f-fb0a3c2d7da6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:41.991754 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f607694-a399-48c3-8c5f-fb0a3c2d7da6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:41.993412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f607694-a399-48c3-8c5f-fb0a3c2d7da6 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:41.995681 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-636a87f9-6547-4ac7-a5ba-5a3512598551 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:41.997255 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-636a87f9-6547-4ac7-a5ba-5a3512598551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:42.006508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:42.015112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-4f607694-a399-48c3-8c5f-fb0a3c2d7da6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:42.015112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:42.015112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:42.015112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:42.015112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:42.015112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:42.015112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:42.015112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:42.015112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:42.015112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:42.015112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:42.015112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:42.015112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:42.015112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:42.015112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:42.016134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:42.016134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:42.016134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:42.016134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:42.016134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:42.016134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:42.016134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:42.016134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:42.016134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:42.016134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:42.016134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:42.016134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:42.016134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:42.016134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:42.016134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:42.016134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:42.016134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:42.020108 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:42.020108 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:42.020108 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:42.020108 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:42.020108 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 454/894] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:42.022026 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f607694-a399-48c3-8c5f-fb0a3c2d7da6 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:42.089443 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 441/895] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 09:04:42.092594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:42.116763 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a87f9-6547-4ac7-a5ba-5a3512598551 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:42.164715 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f607694-a399-48c3-8c5f-fb0a3c2d7da6 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:42.215325 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8500ed-112e-4a06-a0f0-98ed1bd2b2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:42.215924 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8500ed-112e-4a06-a0f0-98ed1bd2b2fb None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:42.216218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8500ed-112e-4a06-a0f0-98ed1bd2b2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:42.216499 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8500ed-112e-4a06-a0f0-98ed1bd2b2fb None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:42.217237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8500ed-112e-4a06-a0f0-98ed1bd2b2fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:42.217547 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8500ed-112e-4a06-a0f0-98ed1bd2b2fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:42.218930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8500ed-112e-4a06-a0f0-98ed1bd2b2fb None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:42.231205 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 455/896] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:42.236594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8500ed-112e-4a06-a0f0-98ed1bd2b2fb None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:42.238776 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a87f9-6547-4ac7-a5ba-5a3512598551 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:42.277492 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2ba09-3d30-4049-ba0a-e0836fe92825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:42.278353 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2ba09-3d30-4049-ba0a-e0836fe92825 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:42.278644 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2ba09-3d30-4049-ba0a-e0836fe92825 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:42.279062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2ba09-3d30-4049-ba0a-e0836fe92825 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:42.279771 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e2ba09-3d30-4049-ba0a-e0836fe92825 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:42.281269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e2ba09-3d30-4049-ba0a-e0836fe92825 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:42.295149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-87e2ba09-3d30-4049-ba0a-e0836fe92825 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 09:04:42.295149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:42.295149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:04:42.295149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:04:42.295149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:04:42.295149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:04:42.295149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:04:42.295149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:42.295149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 09:04:42.295149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:42.295149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:42.295149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:42.295149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:42.295149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:42.295149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:42.295149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:42.296205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:42.296205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:42.296205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:42.296205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:42.296205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:42.296205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:42.296205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:42.296205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:04:42.296205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:04:42.296205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:04:42.296205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:04:42.296205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:42.296205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:42.296205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:42.296205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:42.296205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:42.297134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:42.297134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:42.297134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:42.297134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:42.297134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:42.297134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:42.297134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:42.297134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:42.297134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:42.297134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:42.297134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:42.297134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:04:42.297134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:42.297134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 09:04:42.297134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:42.301341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 442/897] 10.222.0.52 () {56 vars in 1132 bytes} [Thu Jun 23 09:04:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:42.302684 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2ba09-3d30-4049-ba0a-e0836fe92825 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:42.312944 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:42.336921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:42.344140 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8500ed-112e-4a06-a0f0-98ed1bd2b2fb None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:42.385058 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e76ff-5f95-4d39-9c39-424fc168890c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:42.386032 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e76ff-5f95-4d39-9c39-424fc168890c None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:42.386467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e76ff-5f95-4d39-9c39-424fc168890c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:42.386747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e76ff-5f95-4d39-9c39-424fc168890c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:42.399739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e76ff-5f95-4d39-9c39-424fc168890c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:42.401793 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e76ff-5f95-4d39-9c39-424fc168890c None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:42.421329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-bf6e76ff-5f95-4d39-9c39-424fc168890c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 09:04:42.421329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:42.421329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:42.421329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:42.421329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:42.421329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:42.421329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:42.421329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:42.421329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:42.421329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:42.421329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:42.421329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:42.421329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:42.421329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:42.421329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:42.421329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:42.422390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:42.422390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:42.422390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:42.422390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:42.422390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:42.422390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:42.422390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:42.422390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:42.422390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:42.422390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:42.422390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:42.422390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:42.422390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:42.422390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:42.422390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:42.422390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:42.422390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:42.422390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:42.423542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:42.423542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:42.423542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:42.423542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:42.423542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:42.423542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:42.423542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:42.423542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:42.423542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 09:04:42.423542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:42.424153 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 456/898] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jun 23 09:04:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:42.425389 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2ba09-3d30-4049-ba0a-e0836fe92825 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:42.429747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e76ff-5f95-4d39-9c39-424fc168890c None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:42.477581 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a482ed9-536f-45e0-bc08-f2c664acff73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:42.479015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a482ed9-536f-45e0-bc08-f2c664acff73 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:42.479474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a482ed9-536f-45e0-bc08-f2c664acff73 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:42.480413 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a482ed9-536f-45e0-bc08-f2c664acff73 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:42.481571 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a482ed9-536f-45e0-bc08-f2c664acff73 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:42.482049 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a482ed9-536f-45e0-bc08-f2c664acff73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:42.484378 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a482ed9-536f-45e0-bc08-f2c664acff73 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:42.491351 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 443/899] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jun 23 09:04:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:42.497135 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a482ed9-536f-45e0-bc08-f2c664acff73 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:42.559218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e76ff-5f95-4d39-9c39-424fc168890c None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:42.588358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:42.611730 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036ee11-4dbf-4659-ac90-116a94b45a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:42.612790 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036ee11-4dbf-4659-ac90-116a94b45a29 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:42.613290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036ee11-4dbf-4659-ac90-116a94b45a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:42.613811 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036ee11-4dbf-4659-ac90-116a94b45a29 None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:42.614934 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036ee11-4dbf-4659-ac90-116a94b45a29 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:42.615518 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036ee11-4dbf-4659-ac90-116a94b45a29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:42.617148 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036ee11-4dbf-4659-ac90-116a94b45a29 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:42.625559 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a482ed9-536f-45e0-bc08-f2c664acff73 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:42.638194 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 457/900] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jun 23 09:04:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:42.643366 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036ee11-4dbf-4659-ac90-116a94b45a29 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:42.659513 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:42.673976 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24025833-ac9c-41b2-819b-ed0fdcddf031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:42.674679 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24025833-ac9c-41b2-819b-ed0fdcddf031 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:42.675011 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24025833-ac9c-41b2-819b-ed0fdcddf031 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:42.675420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24025833-ac9c-41b2-819b-ed0fdcddf031 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:42.676287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24025833-ac9c-41b2-819b-ed0fdcddf031 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:42.676603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24025833-ac9c-41b2-819b-ed0fdcddf031 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:42.678458 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24025833-ac9c-41b2-819b-ed0fdcddf031 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:42.689910 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 444/901] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jun 23 09:04:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:42.691502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24025833-ac9c-41b2-819b-ed0fdcddf031 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:42.763905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036ee11-4dbf-4659-ac90-116a94b45a29 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:42.806199 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f476f-ebf6-4dbd-8881-66e19380e2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:42.807039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f476f-ebf6-4dbd-8881-66e19380e2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:42.807345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f476f-ebf6-4dbd-8881-66e19380e2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:42.807633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f476f-ebf6-4dbd-8881-66e19380e2a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:42.826810 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24025833-ac9c-41b2-819b-ed0fdcddf031 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:42.871702 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a555dd79-6a61-411b-8a20-f3a5061d5da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:42.872719 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555dd79-6a61-411b-8a20-f3a5061d5da1 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:42.873220 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555dd79-6a61-411b-8a20-f3a5061d5da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:42.874143 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555dd79-6a61-411b-8a20-f3a5061d5da1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:42.875505 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a555dd79-6a61-411b-8a20-f3a5061d5da1 None admin] RBAC: Adding query filter params (interface=public, service_id=8e402a9e5c154daa95fe4d9efe0089bf, region_id=RegionOne) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:42.876256 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a555dd79-6a61-411b-8a20-f3a5061d5da1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:42.878459 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a555dd79-6a61-411b-8a20-f3a5061d5da1 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:42.880899 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:42.891254 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 445/902] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jun 23 09:04:42 2022] GET /identity/v3/endpoints?service_id=8e402a9e5c154daa95fe4d9efe0089bf&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:42.893255 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a555dd79-6a61-411b-8a20-f3a5061d5da1 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:42.894641 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f476f-ebf6-4dbd-8881-66e19380e2a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:42.896155 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f476f-ebf6-4dbd-8881-66e19380e2a9 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:42.913580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:43.006056 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a555dd79-6a61-411b-8a20-f3a5061d5da1 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:43.115524 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 458/903] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:04:42 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 09:04:43.115524 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f476f-ebf6-4dbd-8881-66e19380e2a9 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:43.115815 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21643d-e89e-457e-bc5c-5b73ce9d6862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:43.115815 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21643d-e89e-457e-bc5c-5b73ce9d6862 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:43.115815 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21643d-e89e-457e-bc5c-5b73ce9d6862 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:43.115815 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21643d-e89e-457e-bc5c-5b73ce9d6862 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:43.116109 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21643d-e89e-457e-bc5c-5b73ce9d6862 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:43.116109 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21643d-e89e-457e-bc5c-5b73ce9d6862 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:43.116109 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 446/904] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jun 23 09:04:42 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:04:43.142380 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f476f-ebf6-4dbd-8881-66e19380e2a9 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:43.181688 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27deb40c-5cbf-403f-a29d-6c95cfc6e641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:43.182613 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27deb40c-5cbf-403f-a29d-6c95cfc6e641 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:43.182874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27deb40c-5cbf-403f-a29d-6c95cfc6e641 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:43.183179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27deb40c-5cbf-403f-a29d-6c95cfc6e641 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:43.195706 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27deb40c-5cbf-403f-a29d-6c95cfc6e641 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:43.197297 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27deb40c-5cbf-403f-a29d-6c95cfc6e641 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:43.216840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-27deb40c-5cbf-403f-a29d-6c95cfc6e641 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:43.216840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:43.216840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:43.216840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:43.216840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:43.216840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:43.216840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:43.216840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:43.216840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:43.216840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:43.216840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:43.216840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:43.216840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:43.216840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:43.216840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:43.217866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:43.217866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:43.217866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:43.217866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:43.217866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:43.217866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:43.217866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:43.217866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:43.217866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:43.217866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:43.217866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:43.217866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:43.217866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:43.217866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:43.217866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:43.217866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:43.217866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:43.219781 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:43.219781 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:43.219781 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:43.219781 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:43.219781 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 459/905] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:43.221800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:43.225220 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21643d-e89e-457e-bc5c-5b73ce9d6862 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:43.296269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:43.344526 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21643d-e89e-457e-bc5c-5b73ce9d6862 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:43.379807 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadb304-86ad-40dc-bf7a-cafedc8856dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:43.380807 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadb304-86ad-40dc-bf7a-cafedc8856dd None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:43.381330 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadb304-86ad-40dc-bf7a-cafedc8856dd None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:43.381807 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadb304-86ad-40dc-bf7a-cafedc8856dd None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:43.382727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fadb304-86ad-40dc-bf7a-cafedc8856dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:43.383256 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fadb304-86ad-40dc-bf7a-cafedc8856dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:43.385417 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fadb304-86ad-40dc-bf7a-cafedc8856dd None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:43.398175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee687b36-d8e1-48a3-9018-acef7ea7577f None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:43.398468 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee687b36-d8e1-48a3-9018-acef7ea7577f None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:43.398677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee687b36-d8e1-48a3-9018-acef7ea7577f None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:43.399276 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 447/906] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:43.404454 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 460/907] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:04:43.408684 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba63068-563a-4edc-bf49-5887dbe113cb None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:43.409110 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba63068-563a-4edc-bf49-5887dbe113cb None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:43.409477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba63068-563a-4edc-bf49-5887dbe113cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:43.412089 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadb304-86ad-40dc-bf7a-cafedc8856dd None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:43.432041 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-dba63068-563a-4edc-bf49-5887dbe113cb None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:43.433870 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-dba63068-563a-4edc-bf49-5887dbe113cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:43.507244 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 461/908] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:04:43.530155 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba63068-563a-4edc-bf49-5887dbe113cb None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:43.534707 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadb304-86ad-40dc-bf7a-cafedc8856dd None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:43.565277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:43.577906 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072260e6-58ba-4f7a-96c9-c5fc07a59668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:43.579431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072260e6-58ba-4f7a-96c9-c5fc07a59668 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:43.579431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072260e6-58ba-4f7a-96c9-c5fc07a59668 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:43.579726 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072260e6-58ba-4f7a-96c9-c5fc07a59668 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:43.587970 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072260e6-58ba-4f7a-96c9-c5fc07a59668 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:43.590800 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072260e6-58ba-4f7a-96c9-c5fc07a59668 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:43.607625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-072260e6-58ba-4f7a-96c9-c5fc07a59668 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:43.607625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:43.607625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:43.607625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:43.607625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:43.607625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:43.607625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:43.607625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:43.607625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:43.607625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:43.607625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:43.607625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:43.607625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:43.607625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:43.607625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:43.608714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:43.608714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:43.608714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:43.608714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:43.608714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:43.608714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:43.608714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:43.608714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:43.608714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:43.608714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:43.608714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:43.608714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:43.608714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:43.608714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:43.608714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:43.608714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:43.608714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:43.609884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:43.609884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:43.609884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:43.609884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:43.609884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:43.609884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:43.609884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:43.609884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:43.609884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 448/909] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:43.615598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072260e6-58ba-4f7a-96c9-c5fc07a59668 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:43.656936 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:43.667060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba63068-563a-4edc-bf49-5887dbe113cb None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:43.713773 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a4d61-d2c9-4821-84c9-7f455e4ead91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:43.715508 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a4d61-d2c9-4821-84c9-7f455e4ead91 None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:43.715955 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a4d61-d2c9-4821-84c9-7f455e4ead91 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:43.716378 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a4d61-d2c9-4821-84c9-7f455e4ead91 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:43.717271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a4d61-d2c9-4821-84c9-7f455e4ead91 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:43.718799 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a4d61-d2c9-4821-84c9-7f455e4ead91 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:43.745165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072260e6-58ba-4f7a-96c9-c5fc07a59668 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:43.781456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f5162-2517-41f0-8e29-2cad41e5439d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:43.782126 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f5162-2517-41f0-8e29-2cad41e5439d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:43.782414 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f5162-2517-41f0-8e29-2cad41e5439d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:43.782692 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f5162-2517-41f0-8e29-2cad41e5439d None admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:43.783513 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f5162-2517-41f0-8e29-2cad41e5439d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:43.783823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f5162-2517-41f0-8e29-2cad41e5439d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:43.785255 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f5162-2517-41f0-8e29-2cad41e5439d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:43.802959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 449/910] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:43.805107 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f5162-2517-41f0-8e29-2cad41e5439d None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:43.849513 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:43.861698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 462/911] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jun 23 09:04:43 2022] POST /identity/v3/services => generated 241 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:04:43.920772 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f5162-2517-41f0-8e29-2cad41e5439d None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:43.971983 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5749a7-70ac-40cf-8491-3ddc05e5465c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:43.972658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5749a7-70ac-40cf-8491-3ddc05e5465c None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:43.973042 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5749a7-70ac-40cf-8491-3ddc05e5465c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:43.973326 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5749a7-70ac-40cf-8491-3ddc05e5465c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:43.974179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5749a7-70ac-40cf-8491-3ddc05e5465c None admin] RBAC: Adding query filter params (role.id=367179870e5e4f04b19a345f03c6d26b, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=cfbaba84402d4c31bafcd12133d74c03) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:43.974494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5749a7-70ac-40cf-8491-3ddc05e5465c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:43.976032 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5749a7-70ac-40cf-8491-3ddc05e5465c None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:43.982785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 450/912] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:04:43 2022] GET /identity/v3/role_assignments?user.id=cfbaba84402d4c31bafcd12133d74c03&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=367179870e5e4f04b19a345f03c6d26b => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:44.131412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc24c1-bfb9-4051-8a3d-8eda8c75f118 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:44.131761 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc24c1-bfb9-4051-8a3d-8eda8c75f118 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:44.131991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc24c1-bfb9-4051-8a3d-8eda8c75f118 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:44.133682 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 463/913] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:44.139477 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98c152-b83a-4e07-9e51-b157fe21054b None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:44.139767 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98c152-b83a-4e07-9e51-b157fe21054b None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:44.140051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98c152-b83a-4e07-9e51-b157fe21054b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:44.167047 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-aa98c152-b83a-4e07-9e51-b157fe21054b None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:44.168862 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-aa98c152-b83a-4e07-9e51-b157fe21054b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:44.239950 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 451/914] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:04:44.263545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc24c1-bfb9-4051-8a3d-8eda8c75f118 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:44.374661 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc24c1-bfb9-4051-8a3d-8eda8c75f118 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:44.411391 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad19237-3ba8-4c1f-9450-83445e5898f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:44.412347 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad19237-3ba8-4c1f-9450-83445e5898f2 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:44.412633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad19237-3ba8-4c1f-9450-83445e5898f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:44.412913 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad19237-3ba8-4c1f-9450-83445e5898f2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:44.413568 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad19237-3ba8-4c1f-9450-83445e5898f2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:44.415031 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad19237-3ba8-4c1f-9450-83445e5898f2 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:44.427725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-1ad19237-3ba8-4c1f-9450-83445e5898f2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 09:04:44.427725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:44.427725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:04:44.427725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:04:44.427725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:04:44.427725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:04:44.427725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:04:44.427725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:44.427725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 09:04:44.427725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:44.427725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:44.427725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:44.427725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:44.427725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:44.427725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:44.427725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:44.427725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:44.428831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:44.428831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:44.428831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:44.428831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:44.428831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:44.428831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:44.428831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:04:44.428831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:04:44.428831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:04:44.428831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:04:44.428831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:44.428831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:44.428831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:44.428831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:44.428831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:44.428831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:44.429750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:44.429750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:44.429750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:44.429750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:44.429750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:44.429750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:44.429750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:44.429750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:44.429750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:44.429750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:44.429750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:04:44.429750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:44.429750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 09:04:44.429750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:44.429750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 464/915] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:04:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:44.437755 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98c152-b83a-4e07-9e51-b157fe21054b None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:44.468734 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:44.539136 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98c152-b83a-4e07-9e51-b157fe21054b None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:44.574495 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbdc3b2-663f-476a-80fc-3f9627367318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:44.575532 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbdc3b2-663f-476a-80fc-3f9627367318 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:44.576027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbdc3b2-663f-476a-80fc-3f9627367318 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:44.576510 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbdc3b2-663f-476a-80fc-3f9627367318 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:44.577468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbdc3b2-663f-476a-80fc-3f9627367318 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:44.577982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbdc3b2-663f-476a-80fc-3f9627367318 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:44.579581 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbdc3b2-663f-476a-80fc-3f9627367318 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:44.585305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 452/916] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:04:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:44.591653 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad19237-3ba8-4c1f-9450-83445e5898f2 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:44.614316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:44.702134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad19237-3ba8-4c1f-9450-83445e5898f2 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:44.741320 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b4d7d-2671-4f55-82f7-e0042ea7b959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:44.741991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b4d7d-2671-4f55-82f7-e0042ea7b959 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:44.742276 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b4d7d-2671-4f55-82f7-e0042ea7b959 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:44.742552 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b4d7d-2671-4f55-82f7-e0042ea7b959 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:44.743306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b4d7d-2671-4f55-82f7-e0042ea7b959 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:44.743615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b4d7d-2671-4f55-82f7-e0042ea7b959 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:44.744994 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b4d7d-2671-4f55-82f7-e0042ea7b959 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:44.750614 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 465/917] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jun 23 09:04:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:44.756007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbdc3b2-663f-476a-80fc-3f9627367318 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:44.851871 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:44.860970 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbdc3b2-663f-476a-80fc-3f9627367318 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:44.896141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b856dd4-a80d-4a3b-8851-fbaace3fa3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:44.897321 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b856dd4-a80d-4a3b-8851-fbaace3fa3ef None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:44.897871 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b856dd4-a80d-4a3b-8851-fbaace3fa3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:44.898482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b856dd4-a80d-4a3b-8851-fbaace3fa3ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:44.899787 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b856dd4-a80d-4a3b-8851-fbaace3fa3ef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:44.901192 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b856dd4-a80d-4a3b-8851-fbaace3fa3ef None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:44.978603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 453/918] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jun 23 09:04:44 2022] POST /identity/v3/endpoints => generated 346 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:04:45.252060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee4025-c1f3-4c6d-b5e6-40ea1141ef6f None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:45.252400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee4025-c1f3-4c6d-b5e6-40ea1141ef6f None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:45.252623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee4025-c1f3-4c6d-b5e6-40ea1141ef6f None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:45.254283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 466/919] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:45.259735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ce1e8-4b99-49e2-a063-7ea39dd87f89 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:45.260293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ce1e8-4b99-49e2-a063-7ea39dd87f89 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:45.260781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ce1e8-4b99-49e2-a063-7ea39dd87f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:45.278644 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-a92ce1e8-4b99-49e2-a063-7ea39dd87f89 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:45.280178 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a92ce1e8-4b99-49e2-a063-7ea39dd87f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:45.354526 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 454/920] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 09:04:45.378315 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee4025-c1f3-4c6d-b5e6-40ea1141ef6f None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:45.462204 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:45.482407 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee4025-c1f3-4c6d-b5e6-40ea1141ef6f None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:45.520604 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1b6b6-413d-4e02-8adf-b51181add842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:45.521426 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1b6b6-413d-4e02-8adf-b51181add842 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:45.521713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1b6b6-413d-4e02-8adf-b51181add842 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:45.521987 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1b6b6-413d-4e02-8adf-b51181add842 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:45.522643 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1b6b6-413d-4e02-8adf-b51181add842 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:45.524036 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1b6b6-413d-4e02-8adf-b51181add842 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:45.536770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-c0f1b6b6-413d-4e02-8adf-b51181add842 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 09:04:45.536770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:45.536770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:04:45.536770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:04:45.536770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:04:45.536770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:04:45.536770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:04:45.536770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:45.536770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 09:04:45.536770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:45.536770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:45.536770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:45.536770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:45.536770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:45.536770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:45.536770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:45.537798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:45.537798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:45.537798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:45.537798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:45.537798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:45.537798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:45.537798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:45.537798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:04:45.537798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:04:45.537798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:04:45.537798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:04:45.537798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:45.537798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:45.537798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:45.537798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:45.537798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:45.538699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:45.538699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:45.538699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:45.538699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:45.538699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:45.538699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:45.538699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:45.538699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:45.538699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:45.538699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:45.538699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:45.538699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:04:45.538699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:45.538699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 09:04:45.538699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:45.538699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 467/921] 10.222.0.52 () {56 vars in 1132 bytes} [Thu Jun 23 09:04:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:45.543970 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ce1e8-4b99-49e2-a063-7ea39dd87f89 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:45.625739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:45.647720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ce1e8-4b99-49e2-a063-7ea39dd87f89 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:45.683884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811f639-509c-4e3b-9ae4-aeaeea3f6ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:45.685046 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811f639-509c-4e3b-9ae4-aeaeea3f6ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:45.685542 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811f639-509c-4e3b-9ae4-aeaeea3f6ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:45.686024 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811f639-509c-4e3b-9ae4-aeaeea3f6ca5 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:45.687001 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811f639-509c-4e3b-9ae4-aeaeea3f6ca5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:45.687538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811f639-509c-4e3b-9ae4-aeaeea3f6ca5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:45.689138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811f639-509c-4e3b-9ae4-aeaeea3f6ca5 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:45.694941 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 455/922] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jun 23 09:04:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:45.701136 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1b6b6-413d-4e02-8adf-b51181add842 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:45.808041 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1b6b6-413d-4e02-8adf-b51181add842 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:45.848485 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87014af8-5317-4e06-b437-989e3570d49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:45.849162 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87014af8-5317-4e06-b437-989e3570d49f None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:45.849447 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87014af8-5317-4e06-b437-989e3570d49f None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:45.849722 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87014af8-5317-4e06-b437-989e3570d49f None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:45.850435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87014af8-5317-4e06-b437-989e3570d49f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:45.850740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87014af8-5317-4e06-b437-989e3570d49f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:45.852154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87014af8-5317-4e06-b437-989e3570d49f None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:45.857779 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 468/923] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jun 23 09:04:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:45.863275 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811f639-509c-4e3b-9ae4-aeaeea3f6ca5 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:45.973539 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811f639-509c-4e3b-9ae4-aeaeea3f6ca5 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:46.008708 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad118563-a84f-44c9-badf-630a3ca88d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:46.009384 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad118563-a84f-44c9-badf-630a3ca88d99 None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:46.009670 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad118563-a84f-44c9-badf-630a3ca88d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:46.009948 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad118563-a84f-44c9-badf-630a3ca88d99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:46.010704 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad118563-a84f-44c9-badf-630a3ca88d99 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:46.012648 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad118563-a84f-44c9-badf-630a3ca88d99 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:46.127338 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:46.134359 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 456/924] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jun 23 09:04:45 2022] POST /identity/v3/endpoints => generated 347 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:04:46.164400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3bf30-4203-4556-bdc6-d17b97c1c81a None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:46.164861 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3bf30-4203-4556-bdc6-d17b97c1c81a None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:46.165040 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3bf30-4203-4556-bdc6-d17b97c1c81a None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:46.166582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 469/925] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:46.176505 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fad29-7c86-41ac-aeec-19ae68e0495b None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:46.177102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fad29-7c86-41ac-aeec-19ae68e0495b None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:46.177601 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fad29-7c86-41ac-aeec-19ae68e0495b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:46.194969 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c881a03-c476-4459-a4af-a0108879cab5 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:46.195292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c881a03-c476-4459-a4af-a0108879cab5 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:46.195594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c881a03-c476-4459-a4af-a0108879cab5 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:46.197435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-5c1fad29-7c86-41ac-aeec-19ae68e0495b None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:46.197997 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 470/926] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:04:46.199558 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5c1fad29-7c86-41ac-aeec-19ae68e0495b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:46.204281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628ac24-759e-4479-87a6-9deff5369572 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:46.204733 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628ac24-759e-4479-87a6-9deff5369572 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:46.205024 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628ac24-759e-4479-87a6-9deff5369572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:46.224800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-3628ac24-759e-4479-87a6-9deff5369572 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:46.226036 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-3628ac24-759e-4479-87a6-9deff5369572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:46.229375 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:46.284819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 457/927] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:04:46.310058 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1fad29-7c86-41ac-aeec-19ae68e0495b None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:46.312061 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 471/928] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 09:04:46.329577 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628ac24-759e-4479-87a6-9deff5369572 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:46.431699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1fad29-7c86-41ac-aeec-19ae68e0495b None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:46.437636 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628ac24-759e-4479-87a6-9deff5369572 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:46.462948 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:46.476063 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde30ec-3684-4ab3-8a54-e64415497201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:46.478901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2c060-a81b-40fe-af6c-e36d8517f634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:46.479068 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde30ec-3684-4ab3-8a54-e64415497201 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:46.479068 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde30ec-3684-4ab3-8a54-e64415497201 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:46.479238 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2c060-a81b-40fe-af6c-e36d8517f634 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:46.479325 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde30ec-3684-4ab3-8a54-e64415497201 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:46.479428 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2c060-a81b-40fe-af6c-e36d8517f634 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:46.479428 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2c060-a81b-40fe-af6c-e36d8517f634 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:46.479599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde30ec-3684-4ab3-8a54-e64415497201 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:46.480691 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde30ec-3684-4ab3-8a54-e64415497201 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:46.489853 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf2c060-a81b-40fe-af6c-e36d8517f634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:46.493892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-9bde30ec-3684-4ab3-8a54-e64415497201 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 09:04:46.493892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:46.493892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:04:46.493892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:04:46.493892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:04:46.493892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:04:46.493892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:04:46.493892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:46.493892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 09:04:46.493892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:46.493892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:46.493892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:46.493892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:46.493892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:46.493892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:46.493892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:46.493892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:46.495176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:46.495176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:46.495176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:46.495176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:46.495176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:46.495176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:46.495176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:04:46.495176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:04:46.495176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:04:46.495176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:04:46.495176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:46.495176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:46.495176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:46.495176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:46.495176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:46.495176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:46.496235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:46.496235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:46.496235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:46.496235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:46.496235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:46.496235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:46.496235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:46.496235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:46.496235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:46.496235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:46.496235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:04:46.496235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:46.496235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 09:04:46.496235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:46.503610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-bdf2c060-a81b-40fe-af6c-e36d8517f634 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 09:04:46.503610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:46.503610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:46.503610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:46.503610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:46.503610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:46.503610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:46.503610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:46.503610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:46.503610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:46.503610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:46.503610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:46.503610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:46.503610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:46.503610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:46.503610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:46.504629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:46.504629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:46.504629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:46.504629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:46.504629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:46.504629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:46.504629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:46.504629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:46.504629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:46.504629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:46.504629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:46.504629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:46.504629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:46.504629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:46.504629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:46.504629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:46.505790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:46.505790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:46.505790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:46.505790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:46.505790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:46.505790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:46.505790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:46.505790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:46.505790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:46.505790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:46.505790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 09:04:46.505790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:46.505790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 458/930] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jun 23 09:04:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:46.506532 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 472/930] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jun 23 09:04:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:46.508050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2c060-a81b-40fe-af6c-e36d8517f634 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:46.509308 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde30ec-3684-4ab3-8a54-e64415497201 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:46.515935 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:46.635299 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2c060-a81b-40fe-af6c-e36d8517f634 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:46.635918 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde30ec-3684-4ab3-8a54-e64415497201 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:46.682559 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb6ce2-43bf-40ab-9c83-87e21c0e247f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:46.683999 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb6ce2-43bf-40ab-9c83-87e21c0e247f None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:46.684291 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb6ce2-43bf-40ab-9c83-87e21c0e247f None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:46.684568 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb6ce2-43bf-40ab-9c83-87e21c0e247f None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:46.685287 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb6ce2-43bf-40ab-9c83-87e21c0e247f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:46.685603 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb6ce2-43bf-40ab-9c83-87e21c0e247f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:46.687030 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb6ce2-43bf-40ab-9c83-87e21c0e247f None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:46.688103 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f4b36-b407-4a2c-9f06-177e7dd8cce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:46.688883 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f4b36-b407-4a2c-9f06-177e7dd8cce6 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:46.689270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f4b36-b407-4a2c-9f06-177e7dd8cce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:46.689652 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f4b36-b407-4a2c-9f06-177e7dd8cce6 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:46.690510 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f4b36-b407-4a2c-9f06-177e7dd8cce6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:46.690939 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f4b36-b407-4a2c-9f06-177e7dd8cce6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:46.692424 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f4b36-b407-4a2c-9f06-177e7dd8cce6 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:46.692729 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 473/931] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jun 23 09:04:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:46.698950 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb6ce2-43bf-40ab-9c83-87e21c0e247f None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:46.702198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 459/932] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jun 23 09:04:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:46.707252 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f4b36-b407-4a2c-9f06-177e7dd8cce6 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:46.760237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:46.787471 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:46.825718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f4b36-b407-4a2c-9f06-177e7dd8cce6 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:46.828747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb6ce2-43bf-40ab-9c83-87e21c0e247f None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:46.862289 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e25973-781f-4132-b894-0b6369733d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:46.863260 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e25973-781f-4132-b894-0b6369733d99 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:46.863668 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e25973-781f-4132-b894-0b6369733d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:46.864224 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e25973-781f-4132-b894-0b6369733d99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:46.867370 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7776aa1-fad0-43b1-8d8e-240374d92b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:46.868397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7776aa1-fad0-43b1-8d8e-240374d92b21 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:46.868833 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7776aa1-fad0-43b1-8d8e-240374d92b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:46.869271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7776aa1-fad0-43b1-8d8e-240374d92b21 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:46.870132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7776aa1-fad0-43b1-8d8e-240374d92b21 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:46.870602 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7776aa1-fad0-43b1-8d8e-240374d92b21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:46.872204 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7776aa1-fad0-43b1-8d8e-240374d92b21 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:46.876287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e25973-781f-4132-b894-0b6369733d99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:46.877846 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e25973-781f-4132-b894-0b6369733d99 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:46.878424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 474/933] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jun 23 09:04:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:46.888050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7776aa1-fad0-43b1-8d8e-240374d92b21 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:46.896257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-c4e25973-781f-4132-b894-0b6369733d99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:46.896257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:46.896257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:46.896257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:46.896257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:46.896257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:46.896257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:46.896257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:46.896257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:46.896257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:46.896257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:46.896257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:46.896257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:46.896257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:46.896257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:46.897231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:46.897231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:46.897231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:46.897231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:46.897231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:46.897231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:46.897231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:46.897231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:46.897231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:46.897231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:46.897231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:46.897231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:46.897231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:46.897231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:46.897231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:46.897231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:46.897231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:46.898184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:46.898184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:46.898184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:46.898184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:46.898184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 460/934] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:46.903838 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e25973-781f-4132-b894-0b6369733d99 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:47.009752 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7776aa1-fad0-43b1-8d8e-240374d92b21 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:47.012755 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:47.017350 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e25973-781f-4132-b894-0b6369733d99 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:47.049004 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:47.055283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e4d0f-9703-4ad3-8420-6530913ea615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:47.055904 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8559b5-49b3-472f-8c32-eb342b024c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:47.056097 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e4d0f-9703-4ad3-8420-6530913ea615 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:47.056617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e4d0f-9703-4ad3-8420-6530913ea615 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:47.056910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e4d0f-9703-4ad3-8420-6530913ea615 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:47.057717 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e4d0f-9703-4ad3-8420-6530913ea615 None admin] RBAC: Adding query filter params (interface=public, service_id=d6cb215a37fe4310ad7030678de6c471, region_id=RegionOne) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:47.058083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e4d0f-9703-4ad3-8420-6530913ea615 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:47.059079 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8559b5-49b3-472f-8c32-eb342b024c11 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:47.059518 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8559b5-49b3-472f-8c32-eb342b024c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:47.059953 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8559b5-49b3-472f-8c32-eb342b024c11 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:47.060076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e4d0f-9703-4ad3-8420-6530913ea615 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:47.060963 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8559b5-49b3-472f-8c32-eb342b024c11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:47.061414 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8559b5-49b3-472f-8c32-eb342b024c11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:47.062943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8559b5-49b3-472f-8c32-eb342b024c11 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:47.070943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 475/935] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jun 23 09:04:46 2022] GET /identity/v3/endpoints?service_id=d6cb215a37fe4310ad7030678de6c471&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:47.076939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e4d0f-9703-4ad3-8420-6530913ea615 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:47.082313 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 461/936] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:47.087043 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8559b5-49b3-472f-8c32-eb342b024c11 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:47.206659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e4d0f-9703-4ad3-8420-6530913ea615 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:47.211520 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8559b5-49b3-472f-8c32-eb342b024c11 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:47.246742 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e11ba7-8ee3-4464-9524-ab8810421ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:47.247579 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e11ba7-8ee3-4464-9524-ab8810421ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:47.247866 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e11ba7-8ee3-4464-9524-ab8810421ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:47.248082 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c5f20-775f-4b2a-8e77-f31b9322f4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:47.248224 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e11ba7-8ee3-4464-9524-ab8810421ed0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:47.248853 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c5f20-775f-4b2a-8e77-f31b9322f4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:47.249217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c5f20-775f-4b2a-8e77-f31b9322f4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:47.249496 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c5f20-775f-4b2a-8e77-f31b9322f4e6 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:47.250355 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c5f20-775f-4b2a-8e77-f31b9322f4e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:47.252274 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c5f20-775f-4b2a-8e77-f31b9322f4e6 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:47.255360 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e11ba7-8ee3-4464-9524-ab8810421ed0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:47.256791 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e11ba7-8ee3-4464-9524-ab8810421ed0 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:47.274376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 476/937] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jun 23 09:04:47 2022] GET /identity/v3/services => generated 1497 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:04:47.274376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd343dec-e457-47c2-932f-1d67eb717528 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:47.274376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd343dec-e457-47c2-932f-1d67eb717528 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:47.274376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd343dec-e457-47c2-932f-1d67eb717528 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:47.274376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 477/938] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:47.274376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f301d-9238-4fe5-a863-2e024d9017fd None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:47.274376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f301d-9238-4fe5-a863-2e024d9017fd None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:47.274994 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f301d-9238-4fe5-a863-2e024d9017fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:47.275086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-36e11ba7-8ee3-4464-9524-ab8810421ed0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 09:04:47.275086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:47.275086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:47.275086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:47.275086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:47.275086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:47.275086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:47.275086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:47.275086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:47.275086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:47.275086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:47.275086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:47.275086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:47.275086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:47.275086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:47.275086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:47.276018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:47.276018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:47.276018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:47.276018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:47.276018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:47.276018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:47.276018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:47.276018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:47.276018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:47.276018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:47.276018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:47.276018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:47.276018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:47.276018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:47.276018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:47.276018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:47.276018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:47.276018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:47.277078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:47.277078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:47.277078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:47.277078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:47.277078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:47.277078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:47.277078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:47.277078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:47.277078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 09:04:47.277078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:47.277078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 462/939] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jun 23 09:04:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:47.281121 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e11ba7-8ee3-4464-9524-ab8810421ed0 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:47.297603 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-161f301d-9238-4fe5-a863-2e024d9017fd None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:47.299259 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-161f301d-9238-4fe5-a863-2e024d9017fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:47.304836 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:47.381347 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:47.399444 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 478/940] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:04:47.415087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e11ba7-8ee3-4464-9524-ab8810421ed0 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:47.420030 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f301d-9238-4fe5-a863-2e024d9017fd None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:47.463752 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65013dd-f0c9-40a6-848b-a4057af726b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:47.465357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65013dd-f0c9-40a6-848b-a4057af726b6 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:47.465857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65013dd-f0c9-40a6-848b-a4057af726b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:47.466343 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65013dd-f0c9-40a6-848b-a4057af726b6 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:47.467451 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65013dd-f0c9-40a6-848b-a4057af726b6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:47.467770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65013dd-f0c9-40a6-848b-a4057af726b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:47.469267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65013dd-f0c9-40a6-848b-a4057af726b6 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:47.489750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 463/941] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jun 23 09:04:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:47.495300 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65013dd-f0c9-40a6-848b-a4057af726b6 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:47.540469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f301d-9238-4fe5-a863-2e024d9017fd None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:47.586678 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c7473-c5cb-4660-b57a-23e330ce1963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:47.586891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c7473-c5cb-4660-b57a-23e330ce1963 None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:47.587204 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c7473-c5cb-4660-b57a-23e330ce1963 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:47.587495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c7473-c5cb-4660-b57a-23e330ce1963 None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:47.588211 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c7473-c5cb-4660-b57a-23e330ce1963 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:47.589785 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c7473-c5cb-4660-b57a-23e330ce1963 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:47.591199 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: WARNING keystone.server.flask.common [None req-307c7473-c5cb-4660-b57a-23e330ce1963 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 23 09:04:47.595633 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:47.621892 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65013dd-f0c9-40a6-848b-a4057af726b6 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:47.651850 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.sql.core [None req-307c7473-c5cb-4660-b57a-23e330ce1963 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 09:04:47.651850 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 09:04:47.651850 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [parameters: {'id': '65ffa362a8114f0cb7f62f16b8d351f0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 09:04:47.651850 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80655) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 09:04:47.659559 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92bf58c-3f06-4eef-b965-7feccc922567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:47.660766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bf58c-3f06-4eef-b965-7feccc922567 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:47.661251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bf58c-3f06-4eef-b965-7feccc922567 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:47.661457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-307c7473-c5cb-4660-b57a-23e330ce1963 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 65ffa362a8114f0cb7f62f16b8d351f0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 65ffa362a8114f0cb7f62f16b8d351f0. Jun 23 09:04:47.661457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:47.661457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 09:04:47.661457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 09:04:47.661457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 09:04:47.661457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 09:04:47.661457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 09:04:47.661457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 09:04:47.661457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 09:04:47.661457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application conn.query(q) Jun 23 09:04:47.661457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 09:04:47.661457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 09:04:47.661457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 09:04:47.662307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application result.read() Jun 23 09:04:47.662307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 09:04:47.662307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 09:04:47.662307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 09:04:47.662307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 09:04:47.662307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 09:04:47.662307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 09:04:47.662307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 09:04:47.662307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 09:04:47.662307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 09:04:47.662307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:47.662307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 09:04:47.662307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:47.662307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:47.662307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 09:04:47.662307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 09:04:47.662307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 23 09:04:47.662307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 23 09:04:47.663345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 09:04:47.663345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application next(self.gen) Jun 23 09:04:47.663345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 09:04:47.663345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application yield resource Jun 23 09:04:47.663345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 09:04:47.663345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application next(self.gen) Jun 23 09:04:47.663345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 09:04:47.663345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 09:04:47.663345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 09:04:47.663345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 09:04:47.663345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 09:04:47.663345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise self.value Jun 23 09:04:47.663345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 09:04:47.663345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 09:04:47.663345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 09:04:47.663345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application session.commit() Jun 23 09:04:47.663345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 09:04:47.664836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 09:04:47.664836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 09:04:47.664836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 09:04:47.664836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 09:04:47.664836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self.session.flush() Jun 23 09:04:47.664836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 09:04:47.664836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 09:04:47.664836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 09:04:47.664836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 09:04:47.664836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 09:04:47.664836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application compat.raise_( Jun 23 09:04:47.664836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 09:04:47.664836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception Jun 23 09:04:47.664836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 09:04:47.664836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 09:04:47.664836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 09:04:47.664836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 09:04:47.665789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 09:04:47.665789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 09:04:47.665789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 09:04:47.665789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 09:04:47.665789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 09:04:47.665789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 09:04:47.665789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 09:04:47.665789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 09:04:47.665789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 09:04:47.665789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 09:04:47.665789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 09:04:47.665789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 09:04:47.665789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 09:04:47.665789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 09:04:47.665789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 09:04:47.667889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 09:04:47.667889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 09:04:47.667889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception Jun 23 09:04:47.667889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 09:04:47.667889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 09:04:47.667889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 09:04:47.667889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 09:04:47.667889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 09:04:47.667889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 09:04:47.667889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 09:04:47.667889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application conn.query(q) Jun 23 09:04:47.667889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 09:04:47.667889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 09:04:47.667889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 09:04:47.667889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application result.read() Jun 23 09:04:47.667889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 09:04:47.667889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 09:04:47.670179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 09:04:47.670179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 09:04:47.670179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 09:04:47.670179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 09:04:47.670179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 09:04:47.670179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 09:04:47.670179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 09:04:47.670179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 09:04:47.670179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [parameters: {'id': '65ffa362a8114f0cb7f62f16b8d351f0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 09:04:47.670179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 09:04:47.670179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:47.670179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:47.670179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:47.670179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:47.670179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 23 09:04:47.672420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 23 09:04:47.672420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 09:04:47.672420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 09:04:47.672420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 23 09:04:47.672420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:47.672420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:47.672420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:47.672420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:47.672420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:47.672420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:47.672420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:47.672420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:47.672420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:47.672420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:47.672420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:47.672420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:47.672420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:47.676393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:47.676393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 23 09:04:47.676393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 23 09:04:47.676393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:47.676393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:47.676393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 23 09:04:47.676393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 23 09:04:47.676393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 65ffa362a8114f0cb7f62f16b8d351f0. Jun 23 09:04:47.676393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:47.676393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 479/942] 10.222.0.52 () {60 vars in 1140 bytes} [Thu Jun 23 09:04:47 2022] POST /identity/v3/projects => generated 262 bytes in 244 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:47.676393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c7473-c5cb-4660-b57a-23e330ce1963 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:47.677814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bf58c-3f06-4eef-b965-7feccc922567 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:47.692328 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92bf58c-3f06-4eef-b965-7feccc922567 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:47.694141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92bf58c-3f06-4eef-b965-7feccc922567 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:47.719327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-b92bf58c-3f06-4eef-b965-7feccc922567 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:47.719327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:47.719327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:47.719327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:47.719327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:47.719327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:47.719327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:47.719327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:47.719327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:47.719327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:47.719327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:47.719327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:47.719327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:47.719327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:47.719327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:47.720376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:47.720376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:47.720376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:47.720376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:47.720376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:47.720376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:47.720376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:47.720376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:47.720376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:47.720376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:47.720376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:47.720376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:47.720376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:47.720376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:47.720376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:47.720376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:47.720376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:47.721371 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:47.721371 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:47.721371 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:47.721371 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:47.724345 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 464/943] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:47.726505 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:47.732559 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92bf58c-3f06-4eef-b965-7feccc922567 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:47.803503 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c7473-c5cb-4660-b57a-23e330ce1963 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:47.848941 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afa451-eccb-4c93-a181-387d82e0e8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:47.849913 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afa451-eccb-4c93-a181-387d82e0e8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:47.850301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afa451-eccb-4c93-a181-387d82e0e8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:47.851066 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afa451-eccb-4c93-a181-387d82e0e8f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:47.860234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0afa451-eccb-4c93-a181-387d82e0e8f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:47.862560 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0afa451-eccb-4c93-a181-387d82e0e8f7 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:47.863901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92bf58c-3f06-4eef-b965-7feccc922567 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:47.880507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-d0afa451-eccb-4c93-a181-387d82e0e8f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:47.880507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:47.880507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:47.880507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:47.880507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:47.880507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:47.880507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:47.880507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:47.880507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:47.880507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:47.880507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:47.880507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:47.880507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:47.880507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:47.880507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:47.881504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:47.881504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:47.881504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:47.881504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:47.881504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:47.881504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:47.881504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:47.881504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:47.881504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:47.881504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:47.881504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:47.881504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:47.881504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:47.881504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:47.881504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:47.881504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:47.881504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:47.882471 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:47.882471 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:47.882471 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:47.882471 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:47.882471 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:47.882471 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:47.882471 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:47.882471 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:47.882978 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 480/944] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:47.889655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afa451-eccb-4c93-a181-387d82e0e8f7 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:47.901678 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:47.911977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1791c-ca8c-43f6-b3f7-5750e6fb6041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:47.917082 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1791c-ca8c-43f6-b3f7-5750e6fb6041 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:47.917082 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1791c-ca8c-43f6-b3f7-5750e6fb6041 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:47.917082 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1791c-ca8c-43f6-b3f7-5750e6fb6041 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:47.917082 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c1791c-ca8c-43f6-b3f7-5750e6fb6041 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:47.917082 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c1791c-ca8c-43f6-b3f7-5750e6fb6041 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:47.917082 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c1791c-ca8c-43f6-b3f7-5750e6fb6041 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:47.931069 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 465/945] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:47.943175 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1791c-ca8c-43f6-b3f7-5750e6fb6041 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:48.021178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afa451-eccb-4c93-a181-387d82e0e8f7 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:48.030428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:48.060822 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a7066-2979-4550-b1e8-775bee913cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:48.061892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a7066-2979-4550-b1e8-775bee913cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:48.062071 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1791c-ca8c-43f6-b3f7-5750e6fb6041 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:48.062485 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a7066-2979-4550-b1e8-775bee913cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:48.063071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a7066-2979-4550-b1e8-775bee913cd8 None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:48.063939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a7066-2979-4550-b1e8-775bee913cd8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:48.064422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a7066-2979-4550-b1e8-775bee913cd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:48.066045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a7066-2979-4550-b1e8-775bee913cd8 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:48.089667 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 481/946] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jun 23 09:04:47 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:48.103593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969babc-e2c9-4ad7-99bb-31e15474852a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:48.104817 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969babc-e2c9-4ad7-99bb-31e15474852a None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:48.105787 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969babc-e2c9-4ad7-99bb-31e15474852a None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:48.106214 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969babc-e2c9-4ad7-99bb-31e15474852a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:48.114763 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969babc-e2c9-4ad7-99bb-31e15474852a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:48.116584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969babc-e2c9-4ad7-99bb-31e15474852a None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:48.132667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-6969babc-e2c9-4ad7-99bb-31e15474852a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:48.132667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:48.132667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:48.132667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:48.132667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:48.132667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:48.132667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:48.132667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:48.132667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:48.132667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:48.132667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:48.132667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:48.132667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:48.132667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:48.132667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:48.133879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:48.133879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:48.133879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:48.133879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:48.133879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:48.133879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:48.133879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:48.133879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:48.133879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:48.133879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:48.133879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:48.133879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:48.133879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:48.133879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:48.133879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:48.133879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:48.133879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:48.135096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:48.135096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:48.135096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:48.135096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:48.135096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:48.135096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:48.135096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:48.135096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:48.138142 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 466/947] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:48.144259 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a7066-2979-4550-b1e8-775bee913cd8 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:48.265345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a7066-2979-4550-b1e8-775bee913cd8 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:48.302466 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdbd840-edd3-40ed-8fb3-5316f522c2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:48.303178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdbd840-edd3-40ed-8fb3-5316f522c2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:48.303469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdbd840-edd3-40ed-8fb3-5316f522c2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:48.303738 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdbd840-edd3-40ed-8fb3-5316f522c2a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:48.304541 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdbd840-edd3-40ed-8fb3-5316f522c2a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:48.304859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdbd840-edd3-40ed-8fb3-5316f522c2a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:48.306342 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdbd840-edd3-40ed-8fb3-5316f522c2a8 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:48.320982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 482/948] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:48.321588 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028cba8-8dc9-415f-be0d-db6afa871fbe None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:48.321983 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028cba8-8dc9-415f-be0d-db6afa871fbe None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:48.322369 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028cba8-8dc9-415f-be0d-db6afa871fbe None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:48.323785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 467/949] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:48.325467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdbd840-edd3-40ed-8fb3-5316f522c2a8 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:48.329597 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfda74-8f02-4085-8db5-eb4ed0557773 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:48.329943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfda74-8f02-4085-8db5-eb4ed0557773 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:48.330237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfda74-8f02-4085-8db5-eb4ed0557773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:48.349428 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-74cfda74-8f02-4085-8db5-eb4ed0557773 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:48.350761 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-74cfda74-8f02-4085-8db5-eb4ed0557773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:48.363853 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:48.412260 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:48.442780 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 468/950] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 09:04:48.454333 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdbd840-edd3-40ed-8fb3-5316f522c2a8 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:48.466154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfda74-8f02-4085-8db5-eb4ed0557773 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:48.495503 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066fcd5-d127-4fbe-97ed-e1ca045ef9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:48.497187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066fcd5-d127-4fbe-97ed-e1ca045ef9bb None admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:48.497650 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066fcd5-d127-4fbe-97ed-e1ca045ef9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:48.498198 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066fcd5-d127-4fbe-97ed-e1ca045ef9bb None admin] PATH_INFO: `/v3/projects/93b5040533524222b126bbe93afc56d1/users/cfbaba84402d4c31bafcd12133d74c03/roles/367179870e5e4f04b19a345f03c6d26b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:48.501603 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d066fcd5-d127-4fbe-97ed-e1ca045ef9bb None admin] RBAC: Authorizing `identity:create_grant(project_id=93b5040533524222b126bbe93afc56d1, user_id=cfbaba84402d4c31bafcd12133d74c03, role_id=367179870e5e4f04b19a345f03c6d26b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:48.504044 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d066fcd5-d127-4fbe-97ed-e1ca045ef9bb None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:48.580444 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfda74-8f02-4085-8db5-eb4ed0557773 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:48.586015 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 483/951] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:04:48 2022] PUT /identity/v3/projects/93b5040533524222b126bbe93afc56d1/users/cfbaba84402d4c31bafcd12133d74c03/roles/367179870e5e4f04b19a345f03c6d26b => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:04:48.626115 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edca87a-9eda-437e-91bb-43fc822a6ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:48.627155 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edca87a-9eda-437e-91bb-43fc822a6ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:48.627546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edca87a-9eda-437e-91bb-43fc822a6ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:48.627928 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edca87a-9eda-437e-91bb-43fc822a6ae5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:48.628779 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edca87a-9eda-437e-91bb-43fc822a6ae5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:48.630356 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edca87a-9eda-437e-91bb-43fc822a6ae5 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:48.643669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-5edca87a-9eda-437e-91bb-43fc822a6ae5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 09:04:48.643669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:48.643669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:04:48.643669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:04:48.643669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:04:48.643669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:04:48.643669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:04:48.643669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:48.643669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 09:04:48.643669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:48.643669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:48.643669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:48.643669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:48.643669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:48.643669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:48.643669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:48.643669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:48.645251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:48.645251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:48.645251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:48.645251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:48.645251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:48.645251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:48.645251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:04:48.645251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:04:48.645251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:04:48.645251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:04:48.645251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:48.645251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:48.645251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:48.645251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:48.645251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:48.645251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:48.646237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:48.646237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:48.646237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:48.646237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:48.646237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:48.646237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:48.646237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:48.646237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:48.646237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:48.646237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:48.646237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:04:48.646237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:48.646237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 09:04:48.646237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:48.646237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 469/952] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:48.655306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066fcd5-d127-4fbe-97ed-e1ca045ef9bb None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:48.758814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:48.776373 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066fcd5-d127-4fbe-97ed-e1ca045ef9bb None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:48.812405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f1e49-1c37-45f8-a137-dcdb1d89536c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:48.813095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f1e49-1c37-45f8-a137-dcdb1d89536c None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:48.813385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f1e49-1c37-45f8-a137-dcdb1d89536c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:48.813670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f1e49-1c37-45f8-a137-dcdb1d89536c None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:48.814389 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f1e49-1c37-45f8-a137-dcdb1d89536c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:48.814701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f1e49-1c37-45f8-a137-dcdb1d89536c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:48.816122 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f1e49-1c37-45f8-a137-dcdb1d89536c None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:48.821342 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 484/953] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jun 23 09:04:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:48.827822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edca87a-9eda-437e-91bb-43fc822a6ae5 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:48.874673 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:48.934926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edca87a-9eda-437e-91bb-43fc822a6ae5 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:48.970609 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b265e4-53a1-4a3b-ae96-40143d8a7fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:48.971709 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b265e4-53a1-4a3b-ae96-40143d8a7fbf None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:48.972220 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b265e4-53a1-4a3b-ae96-40143d8a7fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:48.972710 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b265e4-53a1-4a3b-ae96-40143d8a7fbf None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:48.973661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b265e4-53a1-4a3b-ae96-40143d8a7fbf None admin] RBAC: Adding query filter params (type=compute) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:48.974180 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b265e4-53a1-4a3b-ae96-40143d8a7fbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:48.975817 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b265e4-53a1-4a3b-ae96-40143d8a7fbf None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:48.981799 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 470/954] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jun 23 09:04:48 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:49.436916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d22d76-0f92-4954-8018-07a37ef5db04 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:49.437432 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d22d76-0f92-4954-8018-07a37ef5db04 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:49.437432 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d22d76-0f92-4954-8018-07a37ef5db04 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:49.439066 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 485/955] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:49.444268 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd343258-ff04-4928-843a-45d785c62cbd None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:49.444560 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd343258-ff04-4928-843a-45d785c62cbd None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:49.444844 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd343258-ff04-4928-843a-45d785c62cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:49.462209 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-cd343258-ff04-4928-843a-45d785c62cbd None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:49.463423 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-cd343258-ff04-4928-843a-45d785c62cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:49.539046 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 471/956] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:04:49.554966 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d22d76-0f92-4954-8018-07a37ef5db04 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:49.659230 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d22d76-0f92-4954-8018-07a37ef5db04 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:49.695410 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9721ae-13ab-4a72-9bb0-f4573a266aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:49.696563 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9721ae-13ab-4a72-9bb0-f4573a266aff None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:49.697061 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9721ae-13ab-4a72-9bb0-f4573a266aff None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:49.697550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9721ae-13ab-4a72-9bb0-f4573a266aff None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:49.698445 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9721ae-13ab-4a72-9bb0-f4573a266aff None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:49.701227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9721ae-13ab-4a72-9bb0-f4573a266aff None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:49.714350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-df9721ae-13ab-4a72-9bb0-f4573a266aff None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 09:04:49.714350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:49.714350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:04:49.714350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:04:49.714350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:04:49.714350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:04:49.714350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:04:49.714350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:49.714350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 09:04:49.714350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:49.714350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:49.714350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:49.714350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:49.714350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:49.714350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:49.714350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:49.714350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:49.715658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:49.715658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:49.715658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:49.715658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:49.715658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:49.715658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:49.715658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:04:49.715658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:04:49.715658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:04:49.715658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:04:49.715658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:49.715658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:49.715658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:49.715658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:49.715658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:49.715658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:49.716566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:49.716566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:49.716566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:49.716566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:49.716566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:49.716566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:49.716566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:49.716566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:49.716566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:49.716566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:49.716566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:04:49.716566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:49.716566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 09:04:49.716566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:49.719509 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 486/957] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jun 23 09:04:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:49.726951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd343258-ff04-4928-843a-45d785c62cbd None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:49.760863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:49.834709 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd343258-ff04-4928-843a-45d785c62cbd None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:49.871328 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2939a-2ef6-4778-ae1e-bb9f70364058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:49.872293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2939a-2ef6-4778-ae1e-bb9f70364058 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:49.872736 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2939a-2ef6-4778-ae1e-bb9f70364058 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:49.873186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2939a-2ef6-4778-ae1e-bb9f70364058 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:49.874059 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e2939a-2ef6-4778-ae1e-bb9f70364058 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:49.874525 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e2939a-2ef6-4778-ae1e-bb9f70364058 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:49.876119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e2939a-2ef6-4778-ae1e-bb9f70364058 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:49.882120 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 472/958] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jun 23 09:04:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:49.888067 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9721ae-13ab-4a72-9bb0-f4573a266aff None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:49.905059 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:49.991457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9721ae-13ab-4a72-9bb0-f4573a266aff None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:50.028321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ee717-9bf7-474d-857b-4e8aaefa7e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:50.028996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ee717-9bf7-474d-857b-4e8aaefa7e6a None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:50.029283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ee717-9bf7-474d-857b-4e8aaefa7e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:50.029563 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ee717-9bf7-474d-857b-4e8aaefa7e6a None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:50.030273 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ee717-9bf7-474d-857b-4e8aaefa7e6a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:50.030580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ee717-9bf7-474d-857b-4e8aaefa7e6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:50.032036 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ee717-9bf7-474d-857b-4e8aaefa7e6a None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:50.041685 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 487/959] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jun 23 09:04:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:50.043220 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2939a-2ef6-4778-ae1e-bb9f70364058 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:50.146162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:50.149738 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2939a-2ef6-4778-ae1e-bb9f70364058 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:50.192251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a109bac-30d2-4d25-9428-7436286bf16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:50.193363 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a109bac-30d2-4d25-9428-7436286bf16b None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:50.193886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a109bac-30d2-4d25-9428-7436286bf16b None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:50.194370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a109bac-30d2-4d25-9428-7436286bf16b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:50.195396 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a109bac-30d2-4d25-9428-7436286bf16b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:50.196993 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a109bac-30d2-4d25-9428-7436286bf16b None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:50.275889 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 473/960] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jun 23 09:04:50 2022] POST /identity/v3/endpoints => generated 349 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:04:50.304575 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a679ab6-c356-42ab-bb83-0db07c4a673b None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:50.304575 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a679ab6-c356-42ab-bb83-0db07c4a673b None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:50.304575 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a679ab6-c356-42ab-bb83-0db07c4a673b None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:50.306151 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 488/961] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:50.312142 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941435e-304d-467b-a224-d29ea4d1f13f None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:50.312749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941435e-304d-467b-a224-d29ea4d1f13f None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:50.316868 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941435e-304d-467b-a224-d29ea4d1f13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:50.332615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-b941435e-304d-467b-a224-d29ea4d1f13f None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:50.334610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-b941435e-304d-467b-a224-d29ea4d1f13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:50.418728 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 474/962] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 09:04:50.444727 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a679ab6-c356-42ab-bb83-0db07c4a673b None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:50.535770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:50.559538 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a679ab6-c356-42ab-bb83-0db07c4a673b None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:50.596508 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33eff0-0db5-4113-ad56-1a0c73293254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:50.597281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33eff0-0db5-4113-ad56-1a0c73293254 None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:50.597573 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33eff0-0db5-4113-ad56-1a0c73293254 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:50.597857 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33eff0-0db5-4113-ad56-1a0c73293254 None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:50.598578 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33eff0-0db5-4113-ad56-1a0c73293254 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:50.600105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33eff0-0db5-4113-ad56-1a0c73293254 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:50.657701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.sql.core [None req-fb33eff0-0db5-4113-ad56-1a0c73293254 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 09:04:50.657701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 09:04:50.657701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [parameters: {'user_id': '8de0e4cbb2b44da3aa73b47f960b7908', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 09:04:50.657701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80655) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 09:04:50.665878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-fb33eff0-0db5-4113-ad56-1a0c73293254 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 09:04:50.665878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:50.665878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 09:04:50.665878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 09:04:50.665878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 09:04:50.665878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 09:04:50.665878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 09:04:50.665878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 09:04:50.665878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 09:04:50.665878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application conn.query(q) Jun 23 09:04:50.665878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 09:04:50.665878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 09:04:50.665878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 09:04:50.665878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application result.read() Jun 23 09:04:50.665878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 09:04:50.666820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 09:04:50.666820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 09:04:50.666820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 09:04:50.666820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 09:04:50.666820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 09:04:50.666820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 09:04:50.666820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 09:04:50.666820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 09:04:50.666820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:50.666820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 09:04:50.666820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:50.666820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:50.666820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 09:04:50.666820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 09:04:50.666820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 23 09:04:50.666820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 23 09:04:50.666820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 09:04:50.666820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application next(self.gen) Jun 23 09:04:50.668876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 09:04:50.668876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application yield resource Jun 23 09:04:50.668876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 09:04:50.668876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application next(self.gen) Jun 23 09:04:50.668876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 09:04:50.668876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 09:04:50.668876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 09:04:50.668876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 09:04:50.668876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 09:04:50.668876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise self.value Jun 23 09:04:50.668876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 09:04:50.668876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 09:04:50.668876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 09:04:50.668876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application session.commit() Jun 23 09:04:50.668876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 09:04:50.668876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 09:04:50.668876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 09:04:50.670060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 09:04:50.670060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 09:04:50.670060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self.session.flush() Jun 23 09:04:50.670060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 09:04:50.670060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 09:04:50.670060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 09:04:50.670060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 09:04:50.670060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 09:04:50.670060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application compat.raise_( Jun 23 09:04:50.670060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 09:04:50.670060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception Jun 23 09:04:50.670060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 09:04:50.670060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 09:04:50.670060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 09:04:50.670060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 09:04:50.670060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 09:04:50.670060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 09:04:50.671323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 09:04:50.671323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 09:04:50.671323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 23 09:04:50.671323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 23 09:04:50.671323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 09:04:50.671323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 09:04:50.671323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 09:04:50.671323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 09:04:50.671323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 09:04:50.671323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 09:04:50.671323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 09:04:50.671323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 09:04:50.671323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 09:04:50.671323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 09:04:50.671323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 09:04:50.671323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception Jun 23 09:04:50.673263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 09:04:50.673263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 09:04:50.673263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 09:04:50.673263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 09:04:50.673263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 09:04:50.673263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 09:04:50.673263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 09:04:50.673263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application conn.query(q) Jun 23 09:04:50.673263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 09:04:50.673263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 09:04:50.673263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 09:04:50.673263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application result.read() Jun 23 09:04:50.673263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 09:04:50.673263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 09:04:50.673263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 09:04:50.673263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 09:04:50.675440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 09:04:50.675440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 09:04:50.675440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 09:04:50.675440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 09:04:50.675440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 09:04:50.675440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 09:04:50.675440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [parameters: {'user_id': '8de0e4cbb2b44da3aa73b47f960b7908', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 09:04:50.675440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 09:04:50.675440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:50.675440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:50.675440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:50.675440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:50.675440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:50.675440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:50.675440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:50.677381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:50.677381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:50.677381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:50.677381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:50.677381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:50.677381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:50.677381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:50.677381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 23 09:04:50.677381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 23 09:04:50.677381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:50.677381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:50.677381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:50.677381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:50.677381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:50.677381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:50.677381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 23 09:04:50.677381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 23 09:04:50.679798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 23 09:04:50.679798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 23 09:04:50.679798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 09:04:50.679798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 09:04:50.679798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 09:04:50.679798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:50.679798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 489/963] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Jun 23 09:04:50 2022] POST /identity/v3/users => generated 160 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:50.680719 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941435e-304d-467b-a224-d29ea4d1f13f None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:50.759577 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:50.779077 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941435e-304d-467b-a224-d29ea4d1f13f None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:50.817690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf28c39-5932-4e61-90b4-64d58402ba56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:50.819767 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf28c39-5932-4e61-90b4-64d58402ba56 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:50.819767 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf28c39-5932-4e61-90b4-64d58402ba56 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:50.819767 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf28c39-5932-4e61-90b4-64d58402ba56 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:50.826907 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf28c39-5932-4e61-90b4-64d58402ba56 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:50.828659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf28c39-5932-4e61-90b4-64d58402ba56 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:50.844301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-4bf28c39-5932-4e61-90b4-64d58402ba56 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 09:04:50.844301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:50.844301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:50.844301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:50.844301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:50.844301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:50.844301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:50.844301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:50.844301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:50.844301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:50.844301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:50.844301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:50.844301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:50.844301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:50.844301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:50.844301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:50.846724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:50.846724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:50.846724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:50.846724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:50.846724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:50.846724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:50.846724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:50.846724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:50.846724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:50.846724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:50.846724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:50.846724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:50.846724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:50.846724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:50.846724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:50.846724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:50.846724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:50.846724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:50.849165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:50.849165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:50.849165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:50.849165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:50.849165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:50.849165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:50.849165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:50.849165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:50.849165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 09:04:50.849165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:50.849165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 475/964] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:04:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:50.854582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33eff0-0db5-4113-ad56-1a0c73293254 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:50.929009 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e0105-e8cc-492e-a503-9fe182a5cbe8 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:50.929526 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e0105-e8cc-492e-a503-9fe182a5cbe8 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:50.929608 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e0105-e8cc-492e-a503-9fe182a5cbe8 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:50.931065 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 476/965] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:04:50.936728 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcca4bf-9264-4ae9-84ee-f10aa718fcd6 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:50.937271 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcca4bf-9264-4ae9-84ee-f10aa718fcd6 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:50.937748 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcca4bf-9264-4ae9-84ee-f10aa718fcd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:50.955946 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-2bcca4bf-9264-4ae9-84ee-f10aa718fcd6 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:50.957659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-2bcca4bf-9264-4ae9-84ee-f10aa718fcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:50.965032 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33eff0-0db5-4113-ad56-1a0c73293254 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:51.009036 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce9998-d041-4ab9-8c10-64201aac39bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:51.010135 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce9998-d041-4ab9-8c10-64201aac39bc None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:51.010449 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce9998-d041-4ab9-8c10-64201aac39bc None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:51.010735 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce9998-d041-4ab9-8c10-64201aac39bc None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:51.012484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ce9998-d041-4ab9-8c10-64201aac39bc None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:51.012802 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ce9998-d041-4ab9-8c10-64201aac39bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:51.014797 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ce9998-d041-4ab9-8c10-64201aac39bc None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:51.046093 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:51.047470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 477/966] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 09:04:51.061284 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 490/967] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jun 23 09:04:50 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:51.066573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcca4bf-9264-4ae9-84ee-f10aa718fcd6 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:51.164448 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:51.202029 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcca4bf-9264-4ae9-84ee-f10aa718fcd6 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:51.206040 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d02cdd-9609-457a-a66a-4a1862dee386 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:51.206659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d02cdd-9609-457a-a66a-4a1862dee386 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:51.207217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d02cdd-9609-457a-a66a-4a1862dee386 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:51.210633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 491/968] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:04:51.215601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d9d69-a4e6-4793-9ced-e02807dd8914 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:51.217307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d9d69-a4e6-4793-9ced-e02807dd8914 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:51.217796 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d9d69-a4e6-4793-9ced-e02807dd8914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:51.238278 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-739d9d69-a4e6-4793-9ced-e02807dd8914 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:51.240332 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-739d9d69-a4e6-4793-9ced-e02807dd8914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:51.246026 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5a340-701b-4ce3-b161-d1b338dc4858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:51.246855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5a340-701b-4ce3-b161-d1b338dc4858 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:51.247169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5a340-701b-4ce3-b161-d1b338dc4858 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:51.247443 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5a340-701b-4ce3-b161-d1b338dc4858 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:51.262307 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5a340-701b-4ce3-b161-d1b338dc4858 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:51.271762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-94e5a340-701b-4ce3-b161-d1b338dc4858 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 09:04:51.271762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:51.271762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:51.271762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:51.271762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:51.271762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:51.271762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:51.271762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:51.271762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:51.271762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:51.271762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:51.271762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:51.271762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:51.271762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:51.271762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:51.271762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:51.272789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:51.272789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:51.272789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:51.272789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:51.272789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:51.272789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:51.272789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:51.272789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:51.272789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:51.272789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:51.272789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:51.272789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:51.272789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:51.272789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:51.272789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:51.272789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:51.274119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:51.274119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:51.274119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:51.274119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:51.274119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:51.274119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:51.274119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:51.274119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:51.274119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:51.274119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:51.274119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 09:04:51.274119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:51.274119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 478/969] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jun 23 09:04:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:51.279641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5a340-701b-4ce3-b161-d1b338dc4858 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:51.329891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 492/970] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 09:04:51.354121 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d9d69-a4e6-4793-9ced-e02807dd8914 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:51.412361 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5a340-701b-4ce3-b161-d1b338dc4858 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:51.453515 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:51.459191 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eadf161-64c3-4ec5-bff6-0030d3602c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:51.460164 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadf161-64c3-4ec5-bff6-0030d3602c81 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:51.460450 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadf161-64c3-4ec5-bff6-0030d3602c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:51.460729 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadf161-64c3-4ec5-bff6-0030d3602c81 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:51.461519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eadf161-64c3-4ec5-bff6-0030d3602c81 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:51.461836 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eadf161-64c3-4ec5-bff6-0030d3602c81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:51.463335 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eadf161-64c3-4ec5-bff6-0030d3602c81 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:51.473464 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 479/971] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jun 23 09:04:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:51.478299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d9d69-a4e6-4793-9ced-e02807dd8914 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:51.478943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eadf161-64c3-4ec5-bff6-0030d3602c81 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:51.511362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:51.520271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73568c5b-d138-4e8c-abb6-b4669304dbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:51.520950 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73568c5b-d138-4e8c-abb6-b4669304dbe2 None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:51.521240 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73568c5b-d138-4e8c-abb6-b4669304dbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:51.524776 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73568c5b-d138-4e8c-abb6-b4669304dbe2 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:51.524776 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73568c5b-d138-4e8c-abb6-b4669304dbe2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:51.524776 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73568c5b-d138-4e8c-abb6-b4669304dbe2 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:51.583431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eadf161-64c3-4ec5-bff6-0030d3602c81 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:51.617651 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d12a1-26ea-4ebc-8aa2-f0b66ea4d198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:51.618449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d12a1-26ea-4ebc-8aa2-f0b66ea4d198 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:51.618738 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d12a1-26ea-4ebc-8aa2-f0b66ea4d198 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:51.619057 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d12a1-26ea-4ebc-8aa2-f0b66ea4d198 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:51.629055 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d12a1-26ea-4ebc-8aa2-f0b66ea4d198 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:51.630509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d12a1-26ea-4ebc-8aa2-f0b66ea4d198 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:51.648288 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 493/972] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jun 23 09:04:51 2022] POST /identity/v3/services => generated 242 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:04:51.648526 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-830d12a1-26ea-4ebc-8aa2-f0b66ea4d198 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:51.648526 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:51.648526 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:51.648526 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:51.648526 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:51.648526 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:51.648526 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:51.648526 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:51.648526 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:51.648526 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:51.648526 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:51.648526 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:51.648526 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:51.648526 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:51.648526 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:51.650509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:51.650509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:51.650509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:51.650509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:51.650509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:51.650509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:51.650509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:51.650509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:51.650509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:51.650509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:51.650509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:51.650509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:51.650509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:51.650509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:51.650509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:51.650509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:51.650509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:51.651531 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:51.651531 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:51.651531 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:51.651531 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:51.651531 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 480/973] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:51.655814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73568c5b-d138-4e8c-abb6-b4669304dbe2 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:51.777972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73568c5b-d138-4e8c-abb6-b4669304dbe2 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:51.815545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e770a-08d3-4301-a6f9-5ad97891f9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:51.816223 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e770a-08d3-4301-a6f9-5ad97891f9ba None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:51.816510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e770a-08d3-4301-a6f9-5ad97891f9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:51.816788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e770a-08d3-4301-a6f9-5ad97891f9ba None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:51.817516 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e770a-08d3-4301-a6f9-5ad97891f9ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:51.817826 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e770a-08d3-4301-a6f9-5ad97891f9ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:51.819222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e770a-08d3-4301-a6f9-5ad97891f9ba None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:51.832512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 494/974] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:51.838956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d12a1-26ea-4ebc-8aa2-f0b66ea4d198 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:51.893397 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:51.951870 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d12a1-26ea-4ebc-8aa2-f0b66ea4d198 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:51.989535 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91259cba-76d3-471e-9dba-5998ebf8a09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:51.990383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91259cba-76d3-471e-9dba-5998ebf8a09a None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:51.990728 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91259cba-76d3-471e-9dba-5998ebf8a09a None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:51.991034 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91259cba-76d3-471e-9dba-5998ebf8a09a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:51.999128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91259cba-76d3-471e-9dba-5998ebf8a09a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:52.000581 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91259cba-76d3-471e-9dba-5998ebf8a09a None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:52.017381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-91259cba-76d3-471e-9dba-5998ebf8a09a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 09:04:52.017381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:52.017381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:52.017381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:52.017381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:52.017381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:52.017381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:52.017381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:52.017381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:52.017381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:52.017381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:52.017381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:52.017381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:04:52.017381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:04:52.017381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:04:52.017381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:04:52.019394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:52.019394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:52.019394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:04:52.019394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:52.019394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:04:52.019394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:04:52.019394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:52.019394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:52.019394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:52.019394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:52.019394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:52.019394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:52.019394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:52.019394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:52.019394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:52.019394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:52.019394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:52.019394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:52.020795 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:52.020795 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:52.020795 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:04:52.020795 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:04:52.020795 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:04:52.020795 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:04:52.020795 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:04:52.020795 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:04:52.020795 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 09:04:52.020795 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:52.022992 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 481/975] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jun 23 09:04:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:52.027078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e770a-08d3-4301-a6f9-5ad97891f9ba None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:52.076339 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:52.131138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e770a-08d3-4301-a6f9-5ad97891f9ba None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:52.171206 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87843b6-7af2-4f8f-a3de-0f0d5dbd26f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:52.171902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87843b6-7af2-4f8f-a3de-0f0d5dbd26f9 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:52.172191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87843b6-7af2-4f8f-a3de-0f0d5dbd26f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:52.172473 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87843b6-7af2-4f8f-a3de-0f0d5dbd26f9 None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:52.173331 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87843b6-7af2-4f8f-a3de-0f0d5dbd26f9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:52.173643 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87843b6-7af2-4f8f-a3de-0f0d5dbd26f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:52.175104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87843b6-7af2-4f8f-a3de-0f0d5dbd26f9 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:52.199183 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 495/976] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jun 23 09:04:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:52.200023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91259cba-76d3-471e-9dba-5998ebf8a09a None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:52.322038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91259cba-76d3-471e-9dba-5998ebf8a09a None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:52.331167 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:52.363109 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e088944-21d6-43a3-92fd-991583d400b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:52.364208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e088944-21d6-43a3-92fd-991583d400b3 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:52.364643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e088944-21d6-43a3-92fd-991583d400b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:52.364981 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e088944-21d6-43a3-92fd-991583d400b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:52.403068 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe07ad-cce8-4d90-bd7f-cd0fa2f0a212 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:52.403361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe07ad-cce8-4d90-bd7f-cd0fa2f0a212 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:52.403637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe07ad-cce8-4d90-bd7f-cd0fa2f0a212 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:52.405154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 496/977] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:04:52.410087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260197f-8e09-4a5d-8630-311a9f53071d None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:52.410375 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260197f-8e09-4a5d-8630-311a9f53071d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:52.410652 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260197f-8e09-4a5d-8630-311a9f53071d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:52.427488 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-1260197f-8e09-4a5d-8630-311a9f53071d None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:52.428680 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-1260197f-8e09-4a5d-8630-311a9f53071d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:52.442140 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e088944-21d6-43a3-92fd-991583d400b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:52.443640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e088944-21d6-43a3-92fd-991583d400b3 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:52.502232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 497/978] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:52 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 09:04:52.525216 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260197f-8e09-4a5d-8630-311a9f53071d None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:52.555124 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:52.563425 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 482/979] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:04:52 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 09:04:52.564455 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e088944-21d6-43a3-92fd-991583d400b3 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:52.572385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:52.632429 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260197f-8e09-4a5d-8630-311a9f53071d None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:52.666169 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af888b9-17e0-4628-ae78-9d3a3ed0a1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:52.667014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af888b9-17e0-4628-ae78-9d3a3ed0a1ad None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:52.667306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af888b9-17e0-4628-ae78-9d3a3ed0a1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:52.668104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af888b9-17e0-4628-ae78-9d3a3ed0a1ad None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:52.668296 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e088944-21d6-43a3-92fd-991583d400b3 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:52.668765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af888b9-17e0-4628-ae78-9d3a3ed0a1ad None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:52.670140 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af888b9-17e0-4628-ae78-9d3a3ed0a1ad None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:52.681991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-3af888b9-17e0-4628-ae78-9d3a3ed0a1ad None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 09:04:52.681991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:52.681991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:04:52.681991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:04:52.681991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:04:52.681991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:04:52.681991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:04:52.681991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:52.681991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 09:04:52.681991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:52.681991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:52.681991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:52.681991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:52.681991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:52.681991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:52.681991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:52.681991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:52.683105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:52.683105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:52.683105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:52.683105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:52.683105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:52.683105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:52.683105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:04:52.683105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:04:52.683105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:04:52.683105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:04:52.683105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:52.683105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:52.683105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:52.683105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:52.683105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:52.683105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:52.684085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:52.684085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:52.684085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:52.684085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:52.684085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:52.684085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:52.684085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:52.684085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:52.684085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:52.684085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:52.684085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:04:52.684085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:52.684085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 09:04:52.684085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:52.684085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 498/980] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jun 23 09:04:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:52.689064 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af888b9-17e0-4628-ae78-9d3a3ed0a1ad None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:52.707123 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb58e6-fd84-48c0-94dc-357cb3488c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:52.708022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb58e6-fd84-48c0-94dc-357cb3488c2c None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:52.708419 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb58e6-fd84-48c0-94dc-357cb3488c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:52.708715 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb58e6-fd84-48c0-94dc-357cb3488c2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:52.719930 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb58e6-fd84-48c0-94dc-357cb3488c2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:52.721502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb58e6-fd84-48c0-94dc-357cb3488c2c None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:52.740860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-95eb58e6-fd84-48c0-94dc-357cb3488c2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:52.740860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:52.740860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:52.740860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:52.740860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:52.740860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:52.740860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:52.740860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:52.740860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:52.740860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:52.740860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:52.740860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:52.740860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:04:52.740860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:04:52.740860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:04:52.741859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:04:52.741859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:52.741859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:52.741859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:52.741859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:52.741859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:52.741859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:52.741859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:52.741859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:52.741859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:52.741859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:52.741859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:52.741859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:52.741859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:52.741859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:52.741859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:52.741859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:52.742836 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:04:52.742836 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:04:52.742836 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:04:52.742836 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:52.742836 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 483/981] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:52.748098 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb58e6-fd84-48c0-94dc-357cb3488c2c None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:52.788874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:52.796028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af888b9-17e0-4628-ae78-9d3a3ed0a1ad None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:52.811436 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:52.831948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf960c-d74e-4d39-92b6-3e3a26ce09dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:52.832657 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf960c-d74e-4d39-92b6-3e3a26ce09dd None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:52.832918 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf960c-d74e-4d39-92b6-3e3a26ce09dd None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:52.833201 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf960c-d74e-4d39-92b6-3e3a26ce09dd None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:52.833919 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf960c-d74e-4d39-92b6-3e3a26ce09dd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:52.834231 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf960c-d74e-4d39-92b6-3e3a26ce09dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:52.835889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf960c-d74e-4d39-92b6-3e3a26ce09dd None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:52.840860 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 499/982] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jun 23 09:04:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:52.846950 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf960c-d74e-4d39-92b6-3e3a26ce09dd None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:52.855681 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb58e6-fd84-48c0-94dc-357cb3488c2c None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:52.890344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0213b-a552-4248-b164-fdfb804eb453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:52.891005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0213b-a552-4248-b164-fdfb804eb453 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:52.891255 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0213b-a552-4248-b164-fdfb804eb453 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:52.891536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0213b-a552-4248-b164-fdfb804eb453 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:52.892265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0213b-a552-4248-b164-fdfb804eb453 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:52.892577 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0213b-a552-4248-b164-fdfb804eb453 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:52.893958 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0213b-a552-4248-b164-fdfb804eb453 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:52.905881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 484/983] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:52.911190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0213b-a552-4248-b164-fdfb804eb453 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:52.944520 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf960c-d74e-4d39-92b6-3e3a26ce09dd None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:52.979649 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbeef3-9e8f-4d32-9c27-57924e24a6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:52.980395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbeef3-9e8f-4d32-9c27-57924e24a6f0 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:52.980686 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbeef3-9e8f-4d32-9c27-57924e24a6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:52.980970 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbeef3-9e8f-4d32-9c27-57924e24a6f0 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:52.981686 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbeef3-9e8f-4d32-9c27-57924e24a6f0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:52.981996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbeef3-9e8f-4d32-9c27-57924e24a6f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:52.983403 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbeef3-9e8f-4d32-9c27-57924e24a6f0 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:52.988569 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 500/984] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jun 23 09:04:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:52.994295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbeef3-9e8f-4d32-9c27-57924e24a6f0 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:53.010678 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0213b-a552-4248-b164-fdfb804eb453 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:53.020298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:53.045823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6067dc4-3092-47a5-a105-6b428ff4169c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:53.046634 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6067dc4-3092-47a5-a105-6b428ff4169c None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:53.046941 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6067dc4-3092-47a5-a105-6b428ff4169c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:53.047227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6067dc4-3092-47a5-a105-6b428ff4169c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:53.054270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6067dc4-3092-47a5-a105-6b428ff4169c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:53.055890 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6067dc4-3092-47a5-a105-6b428ff4169c None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:53.070285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-f6067dc4-3092-47a5-a105-6b428ff4169c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:53.070285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:53.070285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:53.070285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:53.070285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:53.070285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:53.070285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:53.070285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:53.070285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:53.070285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:53.070285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:53.070285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:53.070285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:04:53.070285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:04:53.070285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:04:53.071589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:04:53.071589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:53.071589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:53.071589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:53.071589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:53.071589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:53.071589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:53.071589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:53.071589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:53.071589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:53.071589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:53.071589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:53.071589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:53.071589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:53.071589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:53.071589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:53.071589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:53.072582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:04:53.072582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:04:53.072582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:04:53.072582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:04:53.072582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:04:53.072582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:04:53.072582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:04:53.072582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:53.072582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 485/985] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:53.077603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6067dc4-3092-47a5-a105-6b428ff4169c None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:53.098530 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbeef3-9e8f-4d32-9c27-57924e24a6f0 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:53.128519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:53.133641 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582585f-7649-4aff-b9cf-9658bb7a163c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:53.134318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582585f-7649-4aff-b9cf-9658bb7a163c None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:53.134614 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582585f-7649-4aff-b9cf-9658bb7a163c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:53.134895 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582585f-7649-4aff-b9cf-9658bb7a163c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:53.135740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582585f-7649-4aff-b9cf-9658bb7a163c None admin] RBAC: Adding query filter params (interface=admin, service_id=d6cb215a37fe4310ad7030678de6c471, region_id=RegionOne) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:53.136063 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582585f-7649-4aff-b9cf-9658bb7a163c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:53.137989 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582585f-7649-4aff-b9cf-9658bb7a163c None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:53.143472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 501/986] 10.222.0.52 () {56 vars in 1249 bytes} [Thu Jun 23 09:04:52 2022] GET /identity/v3/endpoints?service_id=d6cb215a37fe4310ad7030678de6c471&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:53.148766 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582585f-7649-4aff-b9cf-9658bb7a163c None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:53.198094 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6067dc4-3092-47a5-a105-6b428ff4169c None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:53.235310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a25b4-1cdc-4bfc-b780-dd1bc29e77e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:53.235992 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a25b4-1cdc-4bfc-b780-dd1bc29e77e6 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:53.236286 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a25b4-1cdc-4bfc-b780-dd1bc29e77e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:53.236568 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a25b4-1cdc-4bfc-b780-dd1bc29e77e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:53.237386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a25b4-1cdc-4bfc-b780-dd1bc29e77e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:53.237683 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a25b4-1cdc-4bfc-b780-dd1bc29e77e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:53.239160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a25b4-1cdc-4bfc-b780-dd1bc29e77e6 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:53.250654 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 486/987] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:04:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:53.255186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582585f-7649-4aff-b9cf-9658bb7a163c None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:53.256824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a25b4-1cdc-4bfc-b780-dd1bc29e77e6 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:53.258548 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:53.292335 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8cdfd-fffe-46bf-9dbc-ecdba65fc39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:53.293539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8cdfd-fffe-46bf-9dbc-ecdba65fc39e None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:53.293841 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8cdfd-fffe-46bf-9dbc-ecdba65fc39e None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:53.294122 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8cdfd-fffe-46bf-9dbc-ecdba65fc39e None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:53.294816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8cdfd-fffe-46bf-9dbc-ecdba65fc39e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:53.296274 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8cdfd-fffe-46bf-9dbc-ecdba65fc39e None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:53.311336 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 502/988] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jun 23 09:04:53 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:04:53.321599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:53.368334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a25b4-1cdc-4bfc-b780-dd1bc29e77e6 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:53.405752 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f09ed9-91cc-4209-ba37-938d8197128a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:53.406364 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f09ed9-91cc-4209-ba37-938d8197128a None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:53.406657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f09ed9-91cc-4209-ba37-938d8197128a None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:53.406951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f09ed9-91cc-4209-ba37-938d8197128a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:53.407814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f09ed9-91cc-4209-ba37-938d8197128a None admin] RBAC: Adding query filter params (role.id=367179870e5e4f04b19a345f03c6d26b, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=cfbaba84402d4c31bafcd12133d74c03) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:53.408132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f09ed9-91cc-4209-ba37-938d8197128a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:53.409642 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f09ed9-91cc-4209-ba37-938d8197128a None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:53.423243 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 487/989] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:04:53 2022] GET /identity/v3/role_assignments?user.id=cfbaba84402d4c31bafcd12133d74c03&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=367179870e5e4f04b19a345f03c6d26b => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:53.723443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587619d6-caf2-4c2c-b2a7-cb6af14cd6ab None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:53.723959 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587619d6-caf2-4c2c-b2a7-cb6af14cd6ab None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:53.723959 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587619d6-caf2-4c2c-b2a7-cb6af14cd6ab None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:53.725461 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 503/990] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:53.730590 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c85320-d4ed-4641-9254-da05aef003cd None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:53.730868 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c85320-d4ed-4641-9254-da05aef003cd None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:53.731172 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c85320-d4ed-4641-9254-da05aef003cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:53.748834 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-94c85320-d4ed-4641-9254-da05aef003cd None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:53.750023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-94c85320-d4ed-4641-9254-da05aef003cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:53.820240 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 488/991] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:04:53.843116 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587619d6-caf2-4c2c-b2a7-cb6af14cd6ab None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:53.944335 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587619d6-caf2-4c2c-b2a7-cb6af14cd6ab None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:53.978554 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3364e19-9654-4997-b8b1-32924294cb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:53.979305 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3364e19-9654-4997-b8b1-32924294cb07 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:53.979593 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3364e19-9654-4997-b8b1-32924294cb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:53.979867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3364e19-9654-4997-b8b1-32924294cb07 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:53.981061 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3364e19-9654-4997-b8b1-32924294cb07 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:53.982417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3364e19-9654-4997-b8b1-32924294cb07 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:53.994028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-f3364e19-9654-4997-b8b1-32924294cb07 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 09:04:53.994028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:53.994028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:04:53.994028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:04:53.994028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:04:53.994028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:04:53.994028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:04:53.994028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:53.994028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 09:04:53.994028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:53.994028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:53.994028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:53.994028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:53.994028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:53.994028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:53.994028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:53.994028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:53.995428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:53.995428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:53.995428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:53.995428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:53.995428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:53.995428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:53.995428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:04:53.995428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:04:53.995428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:04:53.995428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:04:53.995428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:53.995428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:53.995428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:53.995428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:53.995428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:53.995428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:53.997387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:53.997387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:53.997387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:53.997387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:53.997387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:53.997387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:53.997387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:53.997387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:53.997387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:53.997387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:53.997387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:04:53.997387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:53.997387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 09:04:53.997387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:53.999306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 504/992] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:54.001533 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c85320-d4ed-4641-9254-da05aef003cd None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:54.032753 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:54.108836 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c85320-d4ed-4641-9254-da05aef003cd None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:54.147777 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2f783-4965-4512-b0b5-f1011b136b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:54.150985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2f783-4965-4512-b0b5-f1011b136b85 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:54.150985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2f783-4965-4512-b0b5-f1011b136b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:54.150985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2f783-4965-4512-b0b5-f1011b136b85 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:54.150985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2f783-4965-4512-b0b5-f1011b136b85 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:54.150985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2f783-4965-4512-b0b5-f1011b136b85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:54.152689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2f783-4965-4512-b0b5-f1011b136b85 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:54.159020 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 489/993] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jun 23 09:04:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:54.164935 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3364e19-9654-4997-b8b1-32924294cb07 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:54.186843 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:54.266532 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3364e19-9654-4997-b8b1-32924294cb07 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:54.300366 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6439f4-a29e-4ec2-93c7-fdcee8e1176c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:54.301164 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6439f4-a29e-4ec2-93c7-fdcee8e1176c None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:54.301546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6439f4-a29e-4ec2-93c7-fdcee8e1176c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:54.301925 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6439f4-a29e-4ec2-93c7-fdcee8e1176c None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:54.302727 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6439f4-a29e-4ec2-93c7-fdcee8e1176c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:54.303162 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6439f4-a29e-4ec2-93c7-fdcee8e1176c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:54.304629 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6439f4-a29e-4ec2-93c7-fdcee8e1176c None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:54.315002 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 505/994] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jun 23 09:04:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:54.316516 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2f783-4965-4512-b0b5-f1011b136b85 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:54.413903 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:54.420799 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2f783-4965-4512-b0b5-f1011b136b85 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:54.453363 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e114f-dcfd-4294-a065-ce14429d1e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:54.454106 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e114f-dcfd-4294-a065-ce14429d1e86 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:54.454465 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e114f-dcfd-4294-a065-ce14429d1e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:54.454814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e114f-dcfd-4294-a065-ce14429d1e86 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:54.455702 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e114f-dcfd-4294-a065-ce14429d1e86 None admin] RBAC: Adding query filter params (interface=public, service_id=cb2c917e88b94e29bd755754f9971ca0, region_id=RegionOne) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:54.456094 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e114f-dcfd-4294-a065-ce14429d1e86 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:54.457536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e114f-dcfd-4294-a065-ce14429d1e86 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:54.462957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 490/995] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jun 23 09:04:54 2022] GET /identity/v3/endpoints?service_id=cb2c917e88b94e29bd755754f9971ca0&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:54.469441 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6439f4-a29e-4ec2-93c7-fdcee8e1176c None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:54.557371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:54.577647 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6439f4-a29e-4ec2-93c7-fdcee8e1176c None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:54.613088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc2d3e4-23e9-4ab6-92a4-3e9e0fea3391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:54.613851 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2d3e4-23e9-4ab6-92a4-3e9e0fea3391 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:54.614214 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2d3e4-23e9-4ab6-92a4-3e9e0fea3391 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:54.614564 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2d3e4-23e9-4ab6-92a4-3e9e0fea3391 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:54.615348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc2d3e4-23e9-4ab6-92a4-3e9e0fea3391 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:54.616786 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc2d3e4-23e9-4ab6-92a4-3e9e0fea3391 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:54.630954 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 506/996] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jun 23 09:04:54 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:04:55.499856 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86766aa-7d37-4dee-9781-787bb09fdde3 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:55.500422 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86766aa-7d37-4dee-9781-787bb09fdde3 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:55.500422 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86766aa-7d37-4dee-9781-787bb09fdde3 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:55.501984 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 491/997] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:55.507211 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46049a1b-42cc-4c28-8f1d-5a0d9ae4647a None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:55.507632 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46049a1b-42cc-4c28-8f1d-5a0d9ae4647a None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:55.507825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46049a1b-42cc-4c28-8f1d-5a0d9ae4647a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:55.525220 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-46049a1b-42cc-4c28-8f1d-5a0d9ae4647a None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:55.526425 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-46049a1b-42cc-4c28-8f1d-5a0d9ae4647a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:55.581866 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7053bfc-e718-4902-b02c-ce915e4e651b None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:55.582199 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7053bfc-e718-4902-b02c-ce915e4e651b None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:55.582421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7053bfc-e718-4902-b02c-ce915e4e651b None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:55.583939 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 492/998] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:04:55.588798 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158e437-ecc8-4292-991e-12f8361a4440 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:55.589087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158e437-ecc8-4292-991e-12f8361a4440 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:55.589397 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158e437-ecc8-4292-991e-12f8361a4440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:55.596789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 507/999] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:04:55.607515 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-9158e437-ecc8-4292-991e-12f8361a4440 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:55.608937 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-9158e437-ecc8-4292-991e-12f8361a4440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:55.618202 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46049a1b-42cc-4c28-8f1d-5a0d9ae4647a None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:55.678452 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 493/1000] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 09:04:55.702751 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158e437-ecc8-4292-991e-12f8361a4440 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:55.706713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:55.734107 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46049a1b-42cc-4c28-8f1d-5a0d9ae4647a None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:55.770800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee80f37-48e5-4126-835b-51a497076653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:55.772229 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee80f37-48e5-4126-835b-51a497076653 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:55.772229 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee80f37-48e5-4126-835b-51a497076653 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:55.772229 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee80f37-48e5-4126-835b-51a497076653 None admin] PATH_INFO: `/v3/services/image` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:55.772914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee80f37-48e5-4126-835b-51a497076653 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:55.774316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee80f37-48e5-4126-835b-51a497076653 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:55.787582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-cee80f37-48e5-4126-835b-51a497076653 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 09:04:55.787582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:55.787582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:04:55.787582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:04:55.787582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:04:55.787582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:04:55.787582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:04:55.787582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:55.787582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 09:04:55.787582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:55.787582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:55.787582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:55.787582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:55.787582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:55.787582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:55.787582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:55.787582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:55.788916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:55.788916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:55.788916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:55.788916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:55.788916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:55.788916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:55.788916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:04:55.788916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:04:55.788916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:04:55.788916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:04:55.788916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:55.788916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:55.788916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:55.788916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:55.788916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:55.788916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:55.790010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:55.790010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:55.790010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:55.790010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:55.790010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:55.790010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:55.790010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:55.790010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:55.790010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:55.790010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:55.790010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:04:55.790010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:55.790010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 09:04:55.790010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:04:55.790010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 508/1001] 10.222.0.52 () {56 vars in 1105 bytes} [Thu Jun 23 09:04:55 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:04:55.794517 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee80f37-48e5-4126-835b-51a497076653 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:55.823394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:55.833133 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158e437-ecc8-4292-991e-12f8361a4440 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:55.868551 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bcac2a-4258-4c4b-abaa-819272be7b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:55.869354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bcac2a-4258-4c4b-abaa-819272be7b75 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:55.869625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bcac2a-4258-4c4b-abaa-819272be7b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:55.869903 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bcac2a-4258-4c4b-abaa-819272be7b75 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:55.870557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bcac2a-4258-4c4b-abaa-819272be7b75 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:55.872490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bcac2a-4258-4c4b-abaa-819272be7b75 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:55.884315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-65bcac2a-4258-4c4b-abaa-819272be7b75 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 09:04:55.884315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:55.884315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:04:55.884315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:04:55.884315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:04:55.884315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:04:55.884315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:04:55.884315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:55.884315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 09:04:55.884315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:55.884315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:55.884315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:55.884315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:55.884315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:55.884315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:55.884315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:55.884315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:55.885466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:55.885466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:55.885466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:55.885466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:55.885466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:55.885466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:55.885466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:04:55.885466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:04:55.885466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:04:55.885466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:04:55.885466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:55.885466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:55.885466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:55.885466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:55.885466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:55.885466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:55.886952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:55.886952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:55.886952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:55.886952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:55.886952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:55.886952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:55.886952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:55.886952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:55.886952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:55.886952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:55.886952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:04:55.886952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:55.886952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 09:04:55.886952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:55.886952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 494/1002] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jun 23 09:04:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:55.893978 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bcac2a-4258-4c4b-abaa-819272be7b75 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:55.908831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee80f37-48e5-4126-835b-51a497076653 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:55.944420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b00e60-91c8-49ec-b871-31bae410c030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:55.945084 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b00e60-91c8-49ec-b871-31bae410c030 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:55.945370 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b00e60-91c8-49ec-b871-31bae410c030 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:55.945644 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b00e60-91c8-49ec-b871-31bae410c030 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:55.946351 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b00e60-91c8-49ec-b871-31bae410c030 None admin] RBAC: Adding query filter params (name=image) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:55.946660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b00e60-91c8-49ec-b871-31bae410c030 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:55.948045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b00e60-91c8-49ec-b871-31bae410c030 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:55.953543 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 509/1003] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:55.959089 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b00e60-91c8-49ec-b871-31bae410c030 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:55.962162 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:56.004943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bcac2a-4258-4c4b-abaa-819272be7b75 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:56.038986 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc15f7-e3c6-4cec-b673-6f2577c31c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:56.039623 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc15f7-e3c6-4cec-b673-6f2577c31c02 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:56.039911 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc15f7-e3c6-4cec-b673-6f2577c31c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:56.040193 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc15f7-e3c6-4cec-b673-6f2577c31c02 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:56.040909 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc15f7-e3c6-4cec-b673-6f2577c31c02 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:56.041219 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc15f7-e3c6-4cec-b673-6f2577c31c02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:56.043162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc15f7-e3c6-4cec-b673-6f2577c31c02 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:56.048128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 495/1004] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jun 23 09:04:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:56.053593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc15f7-e3c6-4cec-b673-6f2577c31c02 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:56.067226 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b00e60-91c8-49ec-b871-31bae410c030 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:56.076289 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:56.104679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78eda6-7647-49dd-8b87-6440734eac72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:56.105931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78eda6-7647-49dd-8b87-6440734eac72 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:56.106224 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78eda6-7647-49dd-8b87-6440734eac72 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:56.106507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78eda6-7647-49dd-8b87-6440734eac72 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:56.107261 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78eda6-7647-49dd-8b87-6440734eac72 None admin] RBAC: Adding query filter params (type=image) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:56.107580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78eda6-7647-49dd-8b87-6440734eac72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:56.108954 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78eda6-7647-49dd-8b87-6440734eac72 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:56.113783 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 510/1005] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:55 2022] GET /identity/v3/services?type=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:56.209466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc15f7-e3c6-4cec-b673-6f2577c31c02 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:56.247730 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af832d8-e9e3-4f4d-9fc6-6d5f786db0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:56.248415 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af832d8-e9e3-4f4d-9fc6-6d5f786db0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:56.248704 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af832d8-e9e3-4f4d-9fc6-6d5f786db0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:56.248987 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af832d8-e9e3-4f4d-9fc6-6d5f786db0f7 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:56.249719 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af832d8-e9e3-4f4d-9fc6-6d5f786db0f7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:56.250033 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af832d8-e9e3-4f4d-9fc6-6d5f786db0f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:56.251570 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af832d8-e9e3-4f4d-9fc6-6d5f786db0f7 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:56.257367 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 496/1006] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jun 23 09:04:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:04:56.263470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78eda6-7647-49dd-8b87-6440734eac72 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:56.333168 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:56.368156 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78eda6-7647-49dd-8b87-6440734eac72 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:56.403273 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ce85d-8b11-49e5-88bb-ab447f5f9ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:56.403946 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ce85d-8b11-49e5-88bb-ab447f5f9ede None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:56.404239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ce85d-8b11-49e5-88bb-ab447f5f9ede None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:56.404522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ce85d-8b11-49e5-88bb-ab447f5f9ede None admin] PATH_INFO: `/v3/endpoints` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:56.405279 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ce85d-8b11-49e5-88bb-ab447f5f9ede None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:56.406649 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ce85d-8b11-49e5-88bb-ab447f5f9ede None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:56.474995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 511/1007] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jun 23 09:04:56 2022] POST /identity/v3/endpoints => generated 325 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:04:56.663473 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4dc51d-86fd-41f9-8536-eb25238180c6 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:56.664318 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4dc51d-86fd-41f9-8536-eb25238180c6 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:56.664604 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4dc51d-86fd-41f9-8536-eb25238180c6 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:56.666017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 497/1008] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:56.672141 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0c012-debb-4e36-bd1b-c39075d71a27 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:56.672636 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0c012-debb-4e36-bd1b-c39075d71a27 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:56.672733 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0c012-debb-4e36-bd1b-c39075d71a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:56.690465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-71f0c012-debb-4e36-bd1b-c39075d71a27 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:56.691680 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-71f0c012-debb-4e36-bd1b-c39075d71a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:56.770944 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 512/1009] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:56 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:04:56.788778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4dc51d-86fd-41f9-8536-eb25238180c6 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:56.878161 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:56.897099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4dc51d-86fd-41f9-8536-eb25238180c6 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:56.931466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63106e4f-ed96-4b74-906a-289d27fe9184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:56.932175 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63106e4f-ed96-4b74-906a-289d27fe9184 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:56.932464 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63106e4f-ed96-4b74-906a-289d27fe9184 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:56.932741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63106e4f-ed96-4b74-906a-289d27fe9184 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:56.934073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63106e4f-ed96-4b74-906a-289d27fe9184 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:56.935490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63106e4f-ed96-4b74-906a-289d27fe9184 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:56.948519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-63106e4f-ed96-4b74-906a-289d27fe9184 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 09:04:56.948519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:56.948519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:04:56.948519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:04:56.948519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:04:56.948519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:04:56.948519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:04:56.948519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:56.948519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 09:04:56.948519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:56.948519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:04:56.948519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:56.948519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:56.948519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:56.948519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:56.948519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:56.948519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:56.950119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:56.950119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:56.950119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:56.950119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:56.950119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:56.950119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:56.950119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:04:56.950119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:04:56.950119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:04:56.950119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:04:56.950119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:56.950119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:56.950119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:56.950119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:56.950119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:56.950119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:56.951190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:56.951190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:56.951190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:56.951190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:56.951190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:56.951190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:56.951190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:56.951190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:56.951190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:56.951190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:56.951190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:04:56.951190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:04:56.951190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 09:04:56.951190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:56.951190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 498/1010] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:04:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:56.954386 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0c012-debb-4e36-bd1b-c39075d71a27 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:57.006625 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:57.054280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0c012-debb-4e36-bd1b-c39075d71a27 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:57.088772 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47622402-5a8c-4bb8-a3a1-8ec833568fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:57.090009 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47622402-5a8c-4bb8-a3a1-8ec833568fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:57.090302 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47622402-5a8c-4bb8-a3a1-8ec833568fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:57.090583 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47622402-5a8c-4bb8-a3a1-8ec833568fb5 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:57.091325 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47622402-5a8c-4bb8-a3a1-8ec833568fb5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:57.091635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47622402-5a8c-4bb8-a3a1-8ec833568fb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:57.092998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47622402-5a8c-4bb8-a3a1-8ec833568fb5 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:57.102449 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 513/1011] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jun 23 09:04:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:57.104246 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63106e4f-ed96-4b74-906a-289d27fe9184 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:57.209349 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63106e4f-ed96-4b74-906a-289d27fe9184 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:57.246215 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382c909-7a21-4f04-8d14-0108ae977a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:57.246888 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382c909-7a21-4f04-8d14-0108ae977a50 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:57.247214 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382c909-7a21-4f04-8d14-0108ae977a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:57.247490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382c909-7a21-4f04-8d14-0108ae977a50 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:57.248198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382c909-7a21-4f04-8d14-0108ae977a50 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:04:57.248502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382c909-7a21-4f04-8d14-0108ae977a50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:57.249872 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382c909-7a21-4f04-8d14-0108ae977a50 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:57.255002 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 499/1012] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jun 23 09:04:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:57.262953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47622402-5a8c-4bb8-a3a1-8ec833568fb5 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:57.362363 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47622402-5a8c-4bb8-a3a1-8ec833568fb5 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:57.378369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:57.395312 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244453e-a7be-404c-b9ba-afeeca589fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:57.395968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244453e-a7be-404c-b9ba-afeeca589fb8 None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:57.396251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244453e-a7be-404c-b9ba-afeeca589fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:57.396526 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244453e-a7be-404c-b9ba-afeeca589fb8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:57.397267 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244453e-a7be-404c-b9ba-afeeca589fb8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:57.398715 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244453e-a7be-404c-b9ba-afeeca589fb8 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:57.503565 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 514/1013] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jun 23 09:04:57 2022] POST /identity/v3/endpoints => generated 334 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:04:58.170436 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bb778-bc13-478f-bde3-a8cfa55a8b47 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:58.170900 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bb778-bc13-478f-bde3-a8cfa55a8b47 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:58.171047 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bb778-bc13-478f-bde3-a8cfa55a8b47 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:58.172680 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 500/1014] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:58.177891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18fe5b-97de-4759-9f47-6f4ddc0435ab None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:58.178284 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18fe5b-97de-4759-9f47-6f4ddc0435ab None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:58.178486 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18fe5b-97de-4759-9f47-6f4ddc0435ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:58.195459 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-8b18fe5b-97de-4759-9f47-6f4ddc0435ab None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:58.196646 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-8b18fe5b-97de-4759-9f47-6f4ddc0435ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:58.272736 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 515/1015] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:58 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:04:58.295914 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bb778-bc13-478f-bde3-a8cfa55a8b47 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:58.309623 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:58.404044 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bb778-bc13-478f-bde3-a8cfa55a8b47 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:58.437196 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e34bac-9ff0-4900-bf9b-f19fcd98b04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:58.437867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e34bac-9ff0-4900-bf9b-f19fcd98b04b None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:58.438237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e34bac-9ff0-4900-bf9b-f19fcd98b04b None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:58.438529 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e34bac-9ff0-4900-bf9b-f19fcd98b04b None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:58.439291 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e34bac-9ff0-4900-bf9b-f19fcd98b04b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:58.440685 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e34bac-9ff0-4900-bf9b-f19fcd98b04b None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:58.540212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff5e21-9281-4391-94f0-0cb13cfbb1ad None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:58.540506 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff5e21-9281-4391-94f0-0cb13cfbb1ad None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:58.540792 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff5e21-9281-4391-94f0-0cb13cfbb1ad None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:58.542189 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 516/1016] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:58.547029 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cbc6c-9994-4956-bf73-9c773fc1d6b3 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:58.547320 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cbc6c-9994-4956-bf73-9c773fc1d6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:58.547603 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cbc6c-9994-4956-bf73-9c773fc1d6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:58.566938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 501/1017] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jun 23 09:04:58 2022] POST /identity/v3/services => generated 242 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:04:58.567150 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-395cbc6c-9994-4956-bf73-9c773fc1d6b3 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:58.567150 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-395cbc6c-9994-4956-bf73-9c773fc1d6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:58.653146 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 517/1018] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 09:04:58.680311 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e34bac-9ff0-4900-bf9b-f19fcd98b04b None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:58.757360 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:58.788138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e34bac-9ff0-4900-bf9b-f19fcd98b04b None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:58.822313 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f7fd9-470f-4b3e-8f61-93f909b4b0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:58.823141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f7fd9-470f-4b3e-8f61-93f909b4b0ec None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:58.823492 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f7fd9-470f-4b3e-8f61-93f909b4b0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:58.823803 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f7fd9-470f-4b3e-8f61-93f909b4b0ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:58.826103 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f7fd9-470f-4b3e-8f61-93f909b4b0ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:58.826421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f7fd9-470f-4b3e-8f61-93f909b4b0ec None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:58.828237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 502/1019] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:04:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:04:58.833734 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395cbc6c-9994-4956-bf73-9c773fc1d6b3 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:58.861263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:58.934689 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395cbc6c-9994-4956-bf73-9c773fc1d6b3 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:58.969273 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80161d-2d15-49eb-b4a5-3176d2e5459c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:58.969939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80161d-2d15-49eb-b4a5-3176d2e5459c None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:58.970223 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80161d-2d15-49eb-b4a5-3176d2e5459c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:58.970499 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80161d-2d15-49eb-b4a5-3176d2e5459c None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:58.971229 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80161d-2d15-49eb-b4a5-3176d2e5459c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:04:58.972654 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80161d-2d15-49eb-b4a5-3176d2e5459c None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:04:59.081539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 518/1020] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jun 23 09:04:58 2022] POST /identity/v3/projects => generated 311 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:04:59.661687 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7f717-a40e-410c-b9e3-20049e4e459e None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:59.662206 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7f717-a40e-410c-b9e3-20049e4e459e None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:59.662290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7f717-a40e-410c-b9e3-20049e4e459e None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:59.663692 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 503/1021] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:04:59.668853 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d1f7f-8f08-4a89-8b62-12a3ad36e30d None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:59.669238 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d1f7f-8f08-4a89-8b62-12a3ad36e30d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:59.669429 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d1f7f-8f08-4a89-8b62-12a3ad36e30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:59.694094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-182d1f7f-8f08-4a89-8b62-12a3ad36e30d None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:04:59.695511 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-182d1f7f-8f08-4a89-8b62-12a3ad36e30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:04:59.764094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 519/1022] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:04:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:04:59.787610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7f717-a40e-410c-b9e3-20049e4e459e None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:59.908741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7f717-a40e-410c-b9e3-20049e4e459e None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:04:59.941841 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd974b-c392-4332-8b22-dfdb1cd1a2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:04:59.942580 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd974b-c392-4332-8b22-dfdb1cd1a2ce None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:04:59.942861 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd974b-c392-4332-8b22-dfdb1cd1a2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:04:59.943166 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd974b-c392-4332-8b22-dfdb1cd1a2ce None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:04:59.949813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:04:59.954368 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd974b-c392-4332-8b22-dfdb1cd1a2ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:04:59.962584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-36fd974b-c392-4332-8b22-dfdb1cd1a2ce None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 09:04:59.962584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:04:59.962584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:04:59.962584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:04:59.962584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:04:59.962584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:04:59.962584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:04:59.962584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:04:59.962584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:04:59.962584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:04:59.962584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:04:59.962584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:04:59.962584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:04:59.962584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:04:59.962584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:04:59.962584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:04:59.963635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:04:59.963635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:04:59.963635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:04:59.963635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:04:59.963635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:04:59.963635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:04:59.963635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:04:59.963635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:04:59.963635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:04:59.963635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:04:59.963635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:04:59.963635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:04:59.963635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:04:59.963635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:04:59.963635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:04:59.963635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:04:59.965064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:04:59.965064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:04:59.965064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:04:59.965064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:04:59.965064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:04:59.965064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:04:59.965064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:04:59.965064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:04:59.965064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:04:59.965064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:04:59.965064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 09:04:59.965064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:04:59.966372 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 504/1023] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jun 23 09:04:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:04:59.970315 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d1f7f-8f08-4a89-8b62-12a3ad36e30d None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:04:59.978505 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:00.068230 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d1f7f-8f08-4a89-8b62-12a3ad36e30d None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:00.104268 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f3eae-824f-4a75-8819-65f7be2b0f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:00.104946 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f3eae-824f-4a75-8819-65f7be2b0f34 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:00.105814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f3eae-824f-4a75-8819-65f7be2b0f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:00.106094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f3eae-824f-4a75-8819-65f7be2b0f34 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:00.106852 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f3eae-824f-4a75-8819-65f7be2b0f34 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:00.107272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f3eae-824f-4a75-8819-65f7be2b0f34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:00.108949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f3eae-824f-4a75-8819-65f7be2b0f34 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:00.117797 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 520/1024] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jun 23 09:04:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:05:00.123787 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd974b-c392-4332-8b22-dfdb1cd1a2ce None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:00.248104 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd974b-c392-4332-8b22-dfdb1cd1a2ce None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:00.281947 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42de5c-54cf-4b1d-a7cf-816697cb75eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:00.282799 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42de5c-54cf-4b1d-a7cf-816697cb75eb None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:00.283120 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42de5c-54cf-4b1d-a7cf-816697cb75eb None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:00.283401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42de5c-54cf-4b1d-a7cf-816697cb75eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:00.294308 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec42de5c-54cf-4b1d-a7cf-816697cb75eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:00.295804 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec42de5c-54cf-4b1d-a7cf-816697cb75eb None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:00.313749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-ec42de5c-54cf-4b1d-a7cf-816697cb75eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:05:00.313749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:00.313749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:00.313749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:00.313749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:00.313749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:00.313749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:00.313749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:00.313749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:00.313749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:00.313749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:00.313749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:00.313749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:05:00.313749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:05:00.313749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:05:00.314745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:05:00.314745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:00.314745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:00.314745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:00.314745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:00.314745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:00.314745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:00.314745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:00.314745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:00.314745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:00.314745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:00.314745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:00.314745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:00.314745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:00.314745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:00.314745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:00.314745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:00.316044 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:05:00.316044 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:05:00.316044 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:05:00.316044 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:05:00.316044 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 505/1025] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:00.324200 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f3eae-824f-4a75-8819-65f7be2b0f34 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:00.395934 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:00.430678 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f3eae-824f-4a75-8819-65f7be2b0f34 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:00.465367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af7b64-79a2-4a63-b64b-02ae3677b9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:00.466029 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af7b64-79a2-4a63-b64b-02ae3677b9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:00.466323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af7b64-79a2-4a63-b64b-02ae3677b9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:00.466602 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af7b64-79a2-4a63-b64b-02ae3677b9f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:00.467362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af7b64-79a2-4a63-b64b-02ae3677b9f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:00.467671 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af7b64-79a2-4a63-b64b-02ae3677b9f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:00.469049 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af7b64-79a2-4a63-b64b-02ae3677b9f4 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:00.481922 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 521/1026] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:05:00.487404 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42de5c-54cf-4b1d-a7cf-816697cb75eb None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:00.543479 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:00.595729 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42de5c-54cf-4b1d-a7cf-816697cb75eb None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:00.631357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a70cd-2189-471d-b90d-6ff67cba5368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:00.632424 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a70cd-2189-471d-b90d-6ff67cba5368 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:00.632869 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a70cd-2189-471d-b90d-6ff67cba5368 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:00.633302 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a70cd-2189-471d-b90d-6ff67cba5368 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:00.641897 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a70cd-2189-471d-b90d-6ff67cba5368 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:00.643552 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a70cd-2189-471d-b90d-6ff67cba5368 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:00.650865 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0d5a8-a9ff-4a0a-9b54-bddaab0e2148 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:00.651194 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0d5a8-a9ff-4a0a-9b54-bddaab0e2148 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:00.651479 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0d5a8-a9ff-4a0a-9b54-bddaab0e2148 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:00.652888 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 522/1027] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:00.657682 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321ace6-872c-4669-a7e4-8e69139ef541 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:00.658041 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321ace6-872c-4669-a7e4-8e69139ef541 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:00.658394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321ace6-872c-4669-a7e4-8e69139ef541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:00.659007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-4b6a70cd-2189-471d-b90d-6ff67cba5368 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 09:05:00.659007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:00.659007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:00.659007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:00.659007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:00.659007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:00.659007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:00.659007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:00.659007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:00.659007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:00.659007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:00.659007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:00.659007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:05:00.659007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:05:00.659007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:05:00.659007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:05:00.659997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:00.659997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:00.659997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:05:00.659997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:05:00.659997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:05:00.659997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:05:00.659997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:00.659997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:00.659997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:00.659997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:00.659997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:00.659997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:00.659997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:00.659997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:00.659997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:00.659997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:00.659997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:00.659997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:00.661073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:00.661073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:00.661073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:05:00.661073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:05:00.661073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:05:00.661073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:05:00.661073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:05:00.661073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:05:00.661073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 09:05:00.661073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:05:00.661073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 506/1028] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jun 23 09:05:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:00.667254 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a70cd-2189-471d-b90d-6ff67cba5368 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:00.677705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-e321ace6-872c-4669-a7e4-8e69139ef541 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:00.679302 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e321ace6-872c-4669-a7e4-8e69139ef541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:00.753548 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 523/1029] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:05:00.776170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321ace6-872c-4669-a7e4-8e69139ef541 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:00.777223 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a70cd-2189-471d-b90d-6ff67cba5368 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:00.789211 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:00.823093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e527b0-121c-4b4f-96d9-1c0141d85ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:00.826681 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e527b0-121c-4b4f-96d9-1c0141d85ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:00.827508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e527b0-121c-4b4f-96d9-1c0141d85ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:00.827812 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e527b0-121c-4b4f-96d9-1c0141d85ff6 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:00.829061 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e527b0-121c-4b4f-96d9-1c0141d85ff6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:00.829382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e527b0-121c-4b4f-96d9-1c0141d85ff6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:00.831244 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e527b0-121c-4b4f-96d9-1c0141d85ff6 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:00.846965 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:00.850562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 507/1030] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jun 23 09:05:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:05:00.857089 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e527b0-121c-4b4f-96d9-1c0141d85ff6 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:00.890548 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321ace6-872c-4669-a7e4-8e69139ef541 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:00.927322 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263aee6-704c-4ca3-aa9d-8af02e641b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:00.928125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263aee6-704c-4ca3-aa9d-8af02e641b8b None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:00.928415 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263aee6-704c-4ca3-aa9d-8af02e641b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:00.928696 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263aee6-704c-4ca3-aa9d-8af02e641b8b None admin] PATH_INFO: `/v3/services/image` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:00.929351 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a263aee6-704c-4ca3-aa9d-8af02e641b8b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:00.930778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a263aee6-704c-4ca3-aa9d-8af02e641b8b None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:00.943451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-a263aee6-704c-4ca3-aa9d-8af02e641b8b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 09:05:00.943451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:00.943451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:05:00.943451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:05:00.943451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:05:00.943451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:05:00.943451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:05:00.943451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:05:00.943451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 09:05:00.943451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:05:00.943451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:05:00.943451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:05:00.943451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:00.943451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:00.943451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:00.943451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:00.943451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:00.944549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:00.944549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:00.944549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:00.944549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:00.944549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:00.944549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:00.944549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:05:00.944549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:05:00.944549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:05:00.944549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:05:00.944549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:00.944549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:00.944549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:00.944549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:00.944549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:00.944549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:00.945472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:00.945472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:00.945472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:00.945472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:00.945472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:00.945472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:00.945472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:00.945472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:00.945472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:00.945472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:00.945472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:05:00.945472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:05:00.945472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 09:05:00.945472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:05:00.945472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 524/1031] 10.222.0.52 () {56 vars in 1105 bytes} [Thu Jun 23 09:05:00 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:00.951318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263aee6-704c-4ca3-aa9d-8af02e641b8b None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:00.968377 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e527b0-121c-4b4f-96d9-1c0141d85ff6 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:01.003147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55c365-cde7-4cce-8f04-657eb34d693f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:01.003811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55c365-cde7-4cce-8f04-657eb34d693f None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:01.004100 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55c365-cde7-4cce-8f04-657eb34d693f None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:01.004383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55c365-cde7-4cce-8f04-657eb34d693f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:01.039618 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:01.059754 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263aee6-704c-4ca3-aa9d-8af02e641b8b None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:01.072768 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55c365-cde7-4cce-8f04-657eb34d693f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:01.075018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55c365-cde7-4cce-8f04-657eb34d693f None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:01.086398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:01.100307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d66b92-4312-4a5d-b3ab-11dfcddcb09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:01.101182 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d66b92-4312-4a5d-b3ab-11dfcddcb09d None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:01.101584 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d66b92-4312-4a5d-b3ab-11dfcddcb09d None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:01.101947 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d66b92-4312-4a5d-b3ab-11dfcddcb09d None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:01.102833 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d66b92-4312-4a5d-b3ab-11dfcddcb09d None admin] RBAC: Adding query filter params (name=image) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:01.103526 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d66b92-4312-4a5d-b3ab-11dfcddcb09d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:01.105055 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d66b92-4312-4a5d-b3ab-11dfcddcb09d None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:01.116124 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 525/1032] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:05:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:05:01.117065 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d66b92-4312-4a5d-b3ab-11dfcddcb09d None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:01.180766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 508/1033] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:05:00 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 09:05:01.182058 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc77a4c-e380-479f-a13b-ca3eff3d1d05 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:01.182345 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc77a4c-e380-479f-a13b-ca3eff3d1d05 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:01.182621 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc77a4c-e380-479f-a13b-ca3eff3d1d05 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:01.184423 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 509/1034] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:01.186737 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc77a4c-e380-479f-a13b-ca3eff3d1d05 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:01.237237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d66b92-4312-4a5d-b3ab-11dfcddcb09d None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:01.253514 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:01.270891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef1d2b-c972-48f7-9168-9258048f54a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:01.271582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef1d2b-c972-48f7-9168-9258048f54a6 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:01.271871 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef1d2b-c972-48f7-9168-9258048f54a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:01.272152 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef1d2b-c972-48f7-9168-9258048f54a6 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:01.272879 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef1d2b-c972-48f7-9168-9258048f54a6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:01.273179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef1d2b-c972-48f7-9168-9258048f54a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:01.274565 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef1d2b-c972-48f7-9168-9258048f54a6 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:01.279592 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 526/1035] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:05:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:01.280791 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b141e8-0a53-4ca5-b26c-bdf199998066 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:01.281053 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b141e8-0a53-4ca5-b26c-bdf199998066 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:01.281340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b141e8-0a53-4ca5-b26c-bdf199998066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:01.287431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc77a4c-e380-479f-a13b-ca3eff3d1d05 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:01.297244 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-61b141e8-0a53-4ca5-b26c-bdf199998066 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:01.298451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-61b141e8-0a53-4ca5-b26c-bdf199998066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:01.323091 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f8f27-3a95-4c5f-81ee-8d655f0c38e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:01.323933 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f8f27-3a95-4c5f-81ee-8d655f0c38e5 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:01.324218 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f8f27-3a95-4c5f-81ee-8d655f0c38e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:01.324496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f8f27-3a95-4c5f-81ee-8d655f0c38e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:01.334292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f8f27-3a95-4c5f-81ee-8d655f0c38e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:01.335963 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f8f27-3a95-4c5f-81ee-8d655f0c38e5 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:01.344105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:01.353502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-0c3f8f27-3a95-4c5f-81ee-8d655f0c38e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:05:01.353502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:01.353502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:01.353502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:01.353502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:01.353502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:01.353502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:01.353502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:01.353502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:01.353502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:01.353502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:01.353502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:01.353502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:05:01.353502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:05:01.353502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:05:01.354646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:05:01.354646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:01.354646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:01.354646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:01.354646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:01.354646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:01.354646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:01.354646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:01.354646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:01.354646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:01.354646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:01.354646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:01.354646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:01.354646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:01.354646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:01.354646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:01.354646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:01.356546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:05:01.356546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:05:01.356546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:05:01.356546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:05:01.356546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 510/1036] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:05:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:01.356546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f8f27-3a95-4c5f-81ee-8d655f0c38e5 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:01.367755 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 527/1037] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:05:01.368580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b141e8-0a53-4ca5-b26c-bdf199998066 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:01.460550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f8f27-3a95-4c5f-81ee-8d655f0c38e5 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:01.466572 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b141e8-0a53-4ca5-b26c-bdf199998066 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:01.480689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:01.494270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d60b13-3c7d-47c7-858a-34f743770e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:01.494954 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d60b13-3c7d-47c7-858a-34f743770e51 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:01.495249 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d60b13-3c7d-47c7-858a-34f743770e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:01.495533 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d60b13-3c7d-47c7-858a-34f743770e51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:01.496350 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d60b13-3c7d-47c7-858a-34f743770e51 None admin] RBAC: Adding query filter params (interface=public, service_id=7cb62234620141ad965aa819e99c6253, region_id=RegionOne) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:01.496666 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d60b13-3c7d-47c7-858a-34f743770e51 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:01.498041 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d60b13-3c7d-47c7-858a-34f743770e51 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:01.499519 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf45e81-6b15-4a56-8c5b-b2c32a819a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:01.500188 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf45e81-6b15-4a56-8c5b-b2c32a819a64 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:01.500480 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf45e81-6b15-4a56-8c5b-b2c32a819a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:01.500761 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf45e81-6b15-4a56-8c5b-b2c32a819a64 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:01.501518 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf45e81-6b15-4a56-8c5b-b2c32a819a64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:01.501832 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf45e81-6b15-4a56-8c5b-b2c32a819a64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:01.503020 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 511/1038] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jun 23 09:05:01 2022] GET /identity/v3/endpoints?service_id=7cb62234620141ad965aa819e99c6253&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:01.503310 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf45e81-6b15-4a56-8c5b-b2c32a819a64 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:01.503803 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d60b13-3c7d-47c7-858a-34f743770e51 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:01.522741 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 528/1039] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:01.522741 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf45e81-6b15-4a56-8c5b-b2c32a819a64 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:01.561357 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:01.602478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d60b13-3c7d-47c7-858a-34f743770e51 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:01.617904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf45e81-6b15-4a56-8c5b-b2c32a819a64 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:01.636612 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cccb8d1-d32f-42d2-b5cf-b82110bbae28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:01.637396 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccb8d1-d32f-42d2-b5cf-b82110bbae28 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:01.637682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccb8d1-d32f-42d2-b5cf-b82110bbae28 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:01.637986 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccb8d1-d32f-42d2-b5cf-b82110bbae28 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:01.639187 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cccb8d1-d32f-42d2-b5cf-b82110bbae28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:01.640630 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cccb8d1-d32f-42d2-b5cf-b82110bbae28 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:01.642235 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 512/1040] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:05:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:01.643007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cccb8d1-d32f-42d2-b5cf-b82110bbae28 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:01.655652 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc927bac-0233-4d2f-9be3-fa9b752a0f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:01.656322 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc927bac-0233-4d2f-9be3-fa9b752a0f8d None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:01.656614 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc927bac-0233-4d2f-9be3-fa9b752a0f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:01.656895 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc927bac-0233-4d2f-9be3-fa9b752a0f8d None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:01.657586 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc927bac-0233-4d2f-9be3-fa9b752a0f8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:01.658991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc927bac-0233-4d2f-9be3-fa9b752a0f8d None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:01.671595 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 529/1041] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jun 23 09:05:01 2022] GET /identity/v3/services => generated 1957 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:05:01.671595 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc927bac-0233-4d2f-9be3-fa9b752a0f8d None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:01.704856 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:01.759122 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cccb8d1-d32f-42d2-b5cf-b82110bbae28 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:01.791198 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc927bac-0233-4d2f-9be3-fa9b752a0f8d None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:01.794250 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:01.799456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad4dc3-7d33-43ba-8ebe-74fb2d74374c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:01.800283 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad4dc3-7d33-43ba-8ebe-74fb2d74374c None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:01.800573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad4dc3-7d33-43ba-8ebe-74fb2d74374c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:01.800855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad4dc3-7d33-43ba-8ebe-74fb2d74374c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:01.808279 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad4dc3-7d33-43ba-8ebe-74fb2d74374c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:01.809775 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad4dc3-7d33-43ba-8ebe-74fb2d74374c None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:01.824781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-bfad4dc3-7d33-43ba-8ebe-74fb2d74374c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:05:01.824781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:01.824781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:01.824781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:01.824781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:01.824781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:01.824781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:01.824781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:01.824781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:01.824781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:01.824781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:01.824781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:01.824781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:05:01.824781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:05:01.824781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:05:01.825778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:05:01.825778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:01.825778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:01.825778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:01.825778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:01.825778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:01.825778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:01.825778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:01.825778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:01.825778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:01.825778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:01.825778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:01.825778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:01.825778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:01.825778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:01.825778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:01.825778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:01.826750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:05:01.826750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:05:01.826750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:05:01.826750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:05:01.826750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:05:01.826750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:05:01.826750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:05:01.826750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:05:01.826750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 513/1042] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:05:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:01.832133 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad4dc3-7d33-43ba-8ebe-74fb2d74374c None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:01.833473 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a3c27-2dd8-468a-b1d9-5013761aa99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:01.834615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a3c27-2dd8-468a-b1d9-5013761aa99c None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:01.834615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a3c27-2dd8-468a-b1d9-5013761aa99c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:01.835082 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a3c27-2dd8-468a-b1d9-5013761aa99c None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:01.836091 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a3c27-2dd8-468a-b1d9-5013761aa99c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:01.837857 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a3c27-2dd8-468a-b1d9-5013761aa99c None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:01.937422 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad4dc3-7d33-43ba-8ebe-74fb2d74374c None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:01.949857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:01.970589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b90f5-1eb2-4d67-a724-ca5dfdb6d9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:01.971283 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b90f5-1eb2-4d67-a724-ca5dfdb6d9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:01.971569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b90f5-1eb2-4d67-a724-ca5dfdb6d9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:01.971847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b90f5-1eb2-4d67-a724-ca5dfdb6d9d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:01.983220 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 530/1043] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jun 23 09:05:01 2022] POST /identity/v3/users => generated 285 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:05:01.985047 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b90f5-1eb2-4d67-a724-ca5dfdb6d9d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:01.985047 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b90f5-1eb2-4d67-a724-ca5dfdb6d9d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:01.985047 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b90f5-1eb2-4d67-a724-ca5dfdb6d9d3 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:01.988561 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 514/1044] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:05:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:05:01.995003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a3c27-2dd8-468a-b1d9-5013761aa99c None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:02.108822 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a3c27-2dd8-468a-b1d9-5013761aa99c None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:02.156916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5ae9a-eab5-4ee8-9b28-a1cd1cf8ef88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:02.157675 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5ae9a-eab5-4ee8-9b28-a1cd1cf8ef88 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:02.158022 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5ae9a-eab5-4ee8-9b28-a1cd1cf8ef88 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:02.158362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5ae9a-eab5-4ee8-9b28-a1cd1cf8ef88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:02.159319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e5ae9a-eab5-4ee8-9b28-a1cd1cf8ef88 None admin] RBAC: Adding query filter params (role.id=367179870e5e4f04b19a345f03c6d26b, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=ff025cb706d243778468ae4570374690) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:02.159707 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e5ae9a-eab5-4ee8-9b28-a1cd1cf8ef88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:02.161219 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e5ae9a-eab5-4ee8-9b28-a1cd1cf8ef88 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:02.168105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 531/1045] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:05:01 2022] GET /identity/v3/role_assignments?user.id=ff025cb706d243778468ae4570374690&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=367179870e5e4f04b19a345f03c6d26b => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:03.709372 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ceaa09-7ac0-450c-bddb-b55fb24a6f55 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:03.710184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ceaa09-7ac0-450c-bddb-b55fb24a6f55 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:03.710184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ceaa09-7ac0-450c-bddb-b55fb24a6f55 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:03.711722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 515/1046] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:03.717240 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e76048-abd2-4dc5-a159-2948a6cb394a None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:03.717716 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e76048-abd2-4dc5-a159-2948a6cb394a None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:03.717880 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e76048-abd2-4dc5-a159-2948a6cb394a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:03.734853 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-84e76048-abd2-4dc5-a159-2948a6cb394a None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:03.736084 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-84e76048-abd2-4dc5-a159-2948a6cb394a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:03.753466 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:03.802313 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 532/1047] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:05:03.825372 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ceaa09-7ac0-450c-bddb-b55fb24a6f55 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:03.932766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ceaa09-7ac0-450c-bddb-b55fb24a6f55 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:03.967393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7132992-a80a-4774-898e-ba8e2c1c674c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:03.968103 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7132992-a80a-4774-898e-ba8e2c1c674c None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:03.968386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7132992-a80a-4774-898e-ba8e2c1c674c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:03.968659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7132992-a80a-4774-898e-ba8e2c1c674c None admin] PATH_INFO: `/v3/services/image` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:03.969310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7132992-a80a-4774-898e-ba8e2c1c674c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:03.970684 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7132992-a80a-4774-898e-ba8e2c1c674c None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:03.982170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-d7132992-a80a-4774-898e-ba8e2c1c674c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 09:05:03.982170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:03.982170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:05:03.982170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:05:03.982170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:05:03.982170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:05:03.982170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:05:03.982170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:05:03.982170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 09:05:03.982170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:05:03.982170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:05:03.982170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:05:03.982170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:03.982170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:03.982170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:03.982170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:03.982170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:03.988314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:03.988314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:03.988314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:03.988314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:03.988314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:03.988314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:03.988314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:05:03.988314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:05:03.988314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:05:03.988314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:05:03.988314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:03.988314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:03.988314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:03.988314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:03.988314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:03.988314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:03.989251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:03.989251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:03.989251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:03.989251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:03.989251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:03.989251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:03.989251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:03.989251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:03.989251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:03.989251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:03.989251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:05:03.989251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:05:03.989251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 09:05:03.989251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:05:03.989251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 516/1048] 10.222.0.52 () {56 vars in 1105 bytes} [Thu Jun 23 09:05:03 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:03.990111 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e76048-abd2-4dc5-a159-2948a6cb394a None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:04.087710 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e76048-abd2-4dc5-a159-2948a6cb394a None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:04.121791 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a2c9c0-1615-4361-a673-d4f698761938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:04.122469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2c9c0-1615-4361-a673-d4f698761938 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:04.123033 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2c9c0-1615-4361-a673-d4f698761938 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:04.123146 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2c9c0-1615-4361-a673-d4f698761938 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:04.126251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a2c9c0-1615-4361-a673-d4f698761938 None admin] RBAC: Adding query filter params (name=image) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:04.126251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a2c9c0-1615-4361-a673-d4f698761938 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:04.126471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1cfb9-57ab-4b0f-8071-d42452e3fb89 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:04.126471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1cfb9-57ab-4b0f-8071-d42452e3fb89 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:04.126471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1cfb9-57ab-4b0f-8071-d42452e3fb89 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:04.126734 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a2c9c0-1615-4361-a673-d4f698761938 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:04.127430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 517/1049] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:04.135574 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 533/1050] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:05:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:04.135717 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a886f3-f2c4-4061-a767-09375cd119c2 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:04.135717 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a886f3-f2c4-4061-a767-09375cd119c2 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:04.135717 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a886f3-f2c4-4061-a767-09375cd119c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:04.137200 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a2c9c0-1615-4361-a673-d4f698761938 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:04.154020 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-c6a886f3-f2c4-4061-a767-09375cd119c2 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:04.155278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-c6a886f3-f2c4-4061-a767-09375cd119c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:04.164313 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:04.190902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:04.228994 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 518/1051] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 09:05:04.230393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1d2ec-133d-4411-acb3-6ded20249328 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:04.230678 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1d2ec-133d-4411-acb3-6ded20249328 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:04.230990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1d2ec-133d-4411-acb3-6ded20249328 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:04.232874 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 519/1052] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:04.237471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52444eb-8c63-4fcc-bdd3-b4e535751dcd None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:04.237766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52444eb-8c63-4fcc-bdd3-b4e535751dcd None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:04.238165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52444eb-8c63-4fcc-bdd3-b4e535751dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:04.247606 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a2c9c0-1615-4361-a673-d4f698761938 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:04.254715 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-c52444eb-8c63-4fcc-bdd3-b4e535751dcd None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:04.255976 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-c52444eb-8c63-4fcc-bdd3-b4e535751dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:04.282047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6bc1c-6aa5-4812-9020-b6d8dee78efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:04.282720 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6bc1c-6aa5-4812-9020-b6d8dee78efb None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:04.283044 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6bc1c-6aa5-4812-9020-b6d8dee78efb None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:04.283331 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6bc1c-6aa5-4812-9020-b6d8dee78efb None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:04.284042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d6bc1c-6aa5-4812-9020-b6d8dee78efb None admin] RBAC: Adding query filter params (type=image) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:04.284356 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d6bc1c-6aa5-4812-9020-b6d8dee78efb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:04.285735 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d6bc1c-6aa5-4812-9020-b6d8dee78efb None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:04.290530 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 534/1053] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:05:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:04.291410 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6bc1c-6aa5-4812-9020-b6d8dee78efb None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:04.321668 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 520/1054] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 09:05:04.322346 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52444eb-8c63-4fcc-bdd3-b4e535751dcd None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:04.376265 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:04.393586 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6bc1c-6aa5-4812-9020-b6d8dee78efb None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:04.423284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:04.426423 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6387d3-e614-464d-ae53-4687a9dd8d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:04.427327 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6387d3-e614-464d-ae53-4687a9dd8d45 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:04.427792 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6387d3-e614-464d-ae53-4687a9dd8d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:04.428311 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6387d3-e614-464d-ae53-4687a9dd8d45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:04.429473 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52444eb-8c63-4fcc-bdd3-b4e535751dcd None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:04.438942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6387d3-e614-464d-ae53-4687a9dd8d45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:05:04.446605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-4b6387d3-e614-464d-ae53-4687a9dd8d45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:05:04.446605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:04.446605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:04.446605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:04.446605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:04.446605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:04.446605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:04.446605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:04.446605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:04.446605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:04.446605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:04.446605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:04.446605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:05:04.446605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:05:04.446605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:05:04.446605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:05:04.448207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:05:04.448207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:05:04.448207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:05:04.448207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:05:04.448207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:04.448207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:04.448207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:04.448207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:04.448207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:04.448207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:04.448207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:04.448207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:04.448207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:04.448207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:04.448207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:04.448207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:04.449697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:04.449697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:04.449697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:04.449697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:04.449697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:05:04.449697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:05:04.449697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:05:04.449697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:05:04.449697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:05:04.449697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:05:04.449697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:05:04.449697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:05:04.449697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 535/1055] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:05:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:04.449697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6387d3-e614-464d-ae53-4687a9dd8d45 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:04.464487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5df5d5-2d2a-46d4-ba56-8edf57de3067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:04.465155 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5df5d5-2d2a-46d4-ba56-8edf57de3067 None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:04.465448 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5df5d5-2d2a-46d4-ba56-8edf57de3067 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:04.465729 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5df5d5-2d2a-46d4-ba56-8edf57de3067 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:04.466470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5df5d5-2d2a-46d4-ba56-8edf57de3067 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:04.467879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5df5d5-2d2a-46d4-ba56-8edf57de3067 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:04.549550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6387d3-e614-464d-ae53-4687a9dd8d45 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:04.581664 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae63a9-fda0-47c8-8006-d78c8e089a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:04.582456 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae63a9-fda0-47c8-8006-d78c8e089a87 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:04.582743 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae63a9-fda0-47c8-8006-d78c8e089a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:04.587257 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae63a9-fda0-47c8-8006-d78c8e089a87 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:04.593152 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae63a9-fda0-47c8-8006-d78c8e089a87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:05:04.601290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-f1ae63a9-fda0-47c8-8006-d78c8e089a87 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 09:05:04.601290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:04.601290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:04.601290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:04.601290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:04.601290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:04.601290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:04.601290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:04.601290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:04.601290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:04.601290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:04.601290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:04.601290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:05:04.601290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:05:04.601290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:05:04.601290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:05:04.602465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:05:04.602465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:05:04.602465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:05:04.602465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:05:04.602465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:04.602465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:04.602465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:04.602465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:04.602465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:04.602465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:04.602465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:04.602465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:04.602465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:04.602465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:04.602465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:04.602465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:04.603655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:04.603655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:04.603655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:04.603655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:04.603655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:05:04.603655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:05:04.603655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:05:04.603655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:05:04.603655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:05:04.603655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:05:04.603655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 09:05:04.603655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:05:04.603655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 536/1056] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jun 23 09:05:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:04.603655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae63a9-fda0-47c8-8006-d78c8e089a87 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:04.608663 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:04.621548 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 521/1057] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jun 23 09:05:04 2022] POST /identity/v3/endpoints => generated 327 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:05:04.621548 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5df5d5-2d2a-46d4-ba56-8edf57de3067 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:04.723365 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae63a9-fda0-47c8-8006-d78c8e089a87 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:04.741895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5df5d5-2d2a-46d4-ba56-8edf57de3067 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:04.760843 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c39a83-2fb8-4b5d-93fb-6b0872b51937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:04.762066 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c39a83-2fb8-4b5d-93fb-6b0872b51937 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:04.762356 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c39a83-2fb8-4b5d-93fb-6b0872b51937 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:04.762637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c39a83-2fb8-4b5d-93fb-6b0872b51937 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:04.763433 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c39a83-2fb8-4b5d-93fb-6b0872b51937 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:04.763748 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c39a83-2fb8-4b5d-93fb-6b0872b51937 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:04.765213 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c39a83-2fb8-4b5d-93fb-6b0872b51937 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:04.775369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 537/1058] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:05:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:04.781540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c39a83-2fb8-4b5d-93fb-6b0872b51937 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:04.787312 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa456c26-5ae7-4ad0-9f5b-b3353534fba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:04.787489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa456c26-5ae7-4ad0-9f5b-b3353534fba8 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:04.787777 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa456c26-5ae7-4ad0-9f5b-b3353534fba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:04.788055 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa456c26-5ae7-4ad0-9f5b-b3353534fba8 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:04.788814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa456c26-5ae7-4ad0-9f5b-b3353534fba8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:04.789791 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa456c26-5ae7-4ad0-9f5b-b3353534fba8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:04.790617 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa456c26-5ae7-4ad0-9f5b-b3353534fba8 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:04.800489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 522/1059] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jun 23 09:05:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:04.806257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa456c26-5ae7-4ad0-9f5b-b3353534fba8 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:04.819269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:04.850218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:04.889273 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c39a83-2fb8-4b5d-93fb-6b0872b51937 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:04.911018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa456c26-5ae7-4ad0-9f5b-b3353534fba8 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:04.929312 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f7fb3-1981-4976-b203-e5d4922de953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:04.930182 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f7fb3-1981-4976-b203-e5d4922de953 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:04.930469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f7fb3-1981-4976-b203-e5d4922de953 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:04.930746 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f7fb3-1981-4976-b203-e5d4922de953 None admin] PATH_INFO: `/v3/users/896ee122bcd7464e8f0575668b60948e` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:04.945459 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f7fb3-1981-4976-b203-e5d4922de953 None admin] RBAC: Authorizing `identity:get_user(user_id=896ee122bcd7464e8f0575668b60948e)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:04.946901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f7fb3-1981-4976-b203-e5d4922de953 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:04.947423 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd0b9ce-3b25-47a2-89ce-a95f39d1448b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:04.948229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0b9ce-3b25-47a2-89ce-a95f39d1448b None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:04.948529 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0b9ce-3b25-47a2-89ce-a95f39d1448b None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:04.953352 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0b9ce-3b25-47a2-89ce-a95f39d1448b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:04.953501 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 538/1060] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:05:04 2022] GET /identity/v3/users/896ee122bcd7464e8f0575668b60948e => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:05:04.954315 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f7fb3-1981-4976-b203-e5d4922de953 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:04.964179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd0b9ce-3b25-47a2-89ce-a95f39d1448b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:04.966542 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd0b9ce-3b25-47a2-89ce-a95f39d1448b None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:04.985569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-4cd0b9ce-3b25-47a2-89ce-a95f39d1448b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:05:04.985569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:04.985569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:04.985569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:04.985569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:04.985569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:04.985569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:04.985569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:04.985569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:04.985569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:04.985569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:04.985569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:04.985569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:05:04.985569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:05:04.985569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:05:04.986550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:05:04.986550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:04.986550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:04.986550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:04.986550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:04.986550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:04.986550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:04.986550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:04.986550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:04.986550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:04.986550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:04.986550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:04.986550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:04.986550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:04.986550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:04.986550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:04.986550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:04.987578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:05:04.987578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:05:04.987578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:05:04.987578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:05:04.989153 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 523/1061] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:05:04.994864 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd0b9ce-3b25-47a2-89ce-a95f39d1448b None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:05.062764 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f7fb3-1981-4976-b203-e5d4922de953 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:05.063016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:05.087060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:05.099986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d40c2c-99bf-4311-afc0-902c21e6cffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:05.100975 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d40c2c-99bf-4311-afc0-902c21e6cffb None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:05.101260 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d40c2c-99bf-4311-afc0-902c21e6cffb None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:05.101543 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d40c2c-99bf-4311-afc0-902c21e6cffb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:05.113741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd0b9ce-3b25-47a2-89ce-a95f39d1448b None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:05.150213 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06911a18-34fe-40f8-b9b0-6e0466038960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:05.151255 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06911a18-34fe-40f8-b9b0-6e0466038960 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:05.152307 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06911a18-34fe-40f8-b9b0-6e0466038960 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:05.152832 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06911a18-34fe-40f8-b9b0-6e0466038960 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:05.154018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06911a18-34fe-40f8-b9b0-6e0466038960 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:05.154394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06911a18-34fe-40f8-b9b0-6e0466038960 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:05.155737 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06911a18-34fe-40f8-b9b0-6e0466038960 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:05.173049 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 524/1062] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:05:05.173961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06911a18-34fe-40f8-b9b0-6e0466038960 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:05.180687 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d40c2c-99bf-4311-afc0-902c21e6cffb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:05.182322 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d40c2c-99bf-4311-afc0-902c21e6cffb None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:05.275697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06911a18-34fe-40f8-b9b0-6e0466038960 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:05.295986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 539/1063] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:05:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 09:05:05.301499 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d40c2c-99bf-4311-afc0-902c21e6cffb None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:05.301651 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:05.313760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a563a3a-546f-4064-8b21-6d2b9374da8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:05.314750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a563a3a-546f-4064-8b21-6d2b9374da8c None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:05.315066 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a563a3a-546f-4064-8b21-6d2b9374da8c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:05.315345 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a563a3a-546f-4064-8b21-6d2b9374da8c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:05.322580 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a563a3a-546f-4064-8b21-6d2b9374da8c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:05.324092 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a563a3a-546f-4064-8b21-6d2b9374da8c None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:05.330212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:05.339169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-7a563a3a-546f-4064-8b21-6d2b9374da8c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 09:05:05.339169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:05.339169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:05.339169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:05.339169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:05.339169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:05.339169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:05.339169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:05.339169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:05.339169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:05.339169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:05.339169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:05.339169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:05:05.339169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:05:05.339169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:05:05.339169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:05:05.340412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:05.340412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:05.340412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:05:05.340412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:05:05.340412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:05:05.340412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:05:05.340412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:05.340412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:05.340412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:05.340412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:05.340412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:05.340412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:05.340412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:05.340412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:05.340412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:05.340412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:05.340412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:05.340412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:05.341655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:05.341655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:05.341655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:05:05.341655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:05:05.341655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:05:05.341655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:05:05.341655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:05:05.341655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:05:05.341655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 09:05:05.341655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:05:05.345160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 525/1064] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jun 23 09:05:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:05.347817 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a563a3a-546f-4064-8b21-6d2b9374da8c None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:05.412158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d40c2c-99bf-4311-afc0-902c21e6cffb None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:05.449721 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f71d4-8f14-4a8c-82ec-f83635e6187d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:05.451003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f71d4-8f14-4a8c-82ec-f83635e6187d None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:05.451476 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f71d4-8f14-4a8c-82ec-f83635e6187d None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:05.451985 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f71d4-8f14-4a8c-82ec-f83635e6187d None admin] PATH_INFO: `/v3/projects/b7d8b91570bb4df0942aa7657b75af24` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:05.453365 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f71d4-8f14-4a8c-82ec-f83635e6187d None admin] RBAC: Authorizing `identity:get_project(project_id=b7d8b91570bb4df0942aa7657b75af24)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:05.454372 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a563a3a-546f-4064-8b21-6d2b9374da8c None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:05.455240 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f71d4-8f14-4a8c-82ec-f83635e6187d None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:05.462523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 540/1065] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:05:05 2022] GET /identity/v3/projects/b7d8b91570bb4df0942aa7657b75af24 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:05:05.463585 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f71d4-8f14-4a8c-82ec-f83635e6187d None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:05.488750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592a121-1e6c-431c-b1a9-217fb9d775dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:05.489417 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592a121-1e6c-431c-b1a9-217fb9d775dd None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:05.489706 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592a121-1e6c-431c-b1a9-217fb9d775dd None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:05.489983 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592a121-1e6c-431c-b1a9-217fb9d775dd None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:05.490846 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592a121-1e6c-431c-b1a9-217fb9d775dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:05.491168 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592a121-1e6c-431c-b1a9-217fb9d775dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:05.492606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592a121-1e6c-431c-b1a9-217fb9d775dd None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:05.509765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 526/1066] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jun 23 09:05:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:05.515997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592a121-1e6c-431c-b1a9-217fb9d775dd None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:05.558702 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:05.560482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:05.570370 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f71d4-8f14-4a8c-82ec-f83635e6187d None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:05.604310 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31630966-6e3c-4d2a-978e-9d0efb62385b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:05.605497 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31630966-6e3c-4d2a-978e-9d0efb62385b None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:05.605790 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31630966-6e3c-4d2a-978e-9d0efb62385b None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:05.606069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31630966-6e3c-4d2a-978e-9d0efb62385b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:05.606927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31630966-6e3c-4d2a-978e-9d0efb62385b None admin] RBAC: Adding query filter params (role.id=9cd080b50c6f4ee8a54bf9ef1655bd83, scope.project.id=b7d8b91570bb4df0942aa7657b75af24, user.id=896ee122bcd7464e8f0575668b60948e) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:05.607256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31630966-6e3c-4d2a-978e-9d0efb62385b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:05.608820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31630966-6e3c-4d2a-978e-9d0efb62385b None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:05.618950 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 541/1067] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:05:05 2022] GET /identity/v3/role_assignments?user.id=896ee122bcd7464e8f0575668b60948e&scope.project.id=b7d8b91570bb4df0942aa7657b75af24&role.id=9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:05.619155 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592a121-1e6c-431c-b1a9-217fb9d775dd None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:05.656867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e149a25-2c4d-43f1-a010-37f84dad5040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:05.657683 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e149a25-2c4d-43f1-a010-37f84dad5040 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:05.657967 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e149a25-2c4d-43f1-a010-37f84dad5040 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:05.658242 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e149a25-2c4d-43f1-a010-37f84dad5040 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:05.670637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e149a25-2c4d-43f1-a010-37f84dad5040 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:05.672167 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e149a25-2c4d-43f1-a010-37f84dad5040 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:05.691207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-0e149a25-2c4d-43f1-a010-37f84dad5040 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:05:05.691207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:05.691207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:05.691207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:05.691207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:05.691207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:05.691207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:05.691207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:05.691207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:05.691207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:05.691207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:05.691207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:05.691207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:05:05.691207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:05:05.691207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:05:05.692236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:05:05.692236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:05.692236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:05.692236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:05.692236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:05.692236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:05.692236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:05.692236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:05.692236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:05.692236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:05.692236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:05.692236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:05.692236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:05.692236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:05.692236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:05.692236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:05.692236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:05.693210 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:05:05.693210 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:05:05.693210 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:05:05.693210 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:05:05.693210 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 527/1068] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:05:05.699703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31630966-6e3c-4d2a-978e-9d0efb62385b None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:05.811289 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31630966-6e3c-4d2a-978e-9d0efb62385b None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:05.845175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e9d78-3e52-45ea-9848-cc4ba6e1d27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:05.845765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e9d78-3e52-45ea-9848-cc4ba6e1d27a None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:05.846050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e9d78-3e52-45ea-9848-cc4ba6e1d27a None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:05.846327 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e9d78-3e52-45ea-9848-cc4ba6e1d27a None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:05.847071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e9d78-3e52-45ea-9848-cc4ba6e1d27a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:05.847381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e9d78-3e52-45ea-9848-cc4ba6e1d27a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:05.848734 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e9d78-3e52-45ea-9848-cc4ba6e1d27a None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:05.865219 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 542/1069] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:05.866449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e149a25-2c4d-43f1-a010-37f84dad5040 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:05.971822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:05.983580 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e149a25-2c4d-43f1-a010-37f84dad5040 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:06.019398 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c057b33-acd6-4680-ac30-88fd10bbb3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:06.020198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c057b33-acd6-4680-ac30-88fd10bbb3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:06.020487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c057b33-acd6-4680-ac30-88fd10bbb3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:06.020766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c057b33-acd6-4680-ac30-88fd10bbb3d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:06.028258 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c057b33-acd6-4680-ac30-88fd10bbb3d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:06.029989 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c057b33-acd6-4680-ac30-88fd10bbb3d0 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:06.044703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-9c057b33-acd6-4680-ac30-88fd10bbb3d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:05:06.044703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:06.044703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:06.044703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:06.044703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:06.044703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:06.044703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:06.044703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:06.044703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:06.044703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:06.044703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:06.044703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:06.044703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:05:06.044703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:05:06.044703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:05:06.045816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:05:06.045816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:06.045816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:06.045816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:06.045816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:06.045816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:06.045816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:06.045816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:06.045816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:06.045816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:06.045816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:06.045816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:06.045816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:06.045816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:06.045816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:06.045816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:06.045816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:06.046849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:05:06.046849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:05:06.046849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:05:06.046849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:05:06.046849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:05:06.046849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:05:06.046849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:05:06.046849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:05:06.046849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 528/1070] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:05:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:06.053276 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e9d78-3e52-45ea-9848-cc4ba6e1d27a None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:06.071555 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:06.161767 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e9d78-3e52-45ea-9848-cc4ba6e1d27a None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:06.203688 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cb070-e1cf-4143-bb60-07cee8be5d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:06.204373 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cb070-e1cf-4143-bb60-07cee8be5d30 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:06.204661 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cb070-e1cf-4143-bb60-07cee8be5d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:06.204938 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cb070-e1cf-4143-bb60-07cee8be5d30 None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:06.205732 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cb070-e1cf-4143-bb60-07cee8be5d30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:06.206044 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cb070-e1cf-4143-bb60-07cee8be5d30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:06.207514 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cb070-e1cf-4143-bb60-07cee8be5d30 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:06.220697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 543/1071] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:05:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:05:06.226390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c057b33-acd6-4680-ac30-88fd10bbb3d0 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:06.330589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c057b33-acd6-4680-ac30-88fd10bbb3d0 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:06.364779 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2d3bb-3370-4226-89cd-ce3c5256060e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:06.365734 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2d3bb-3370-4226-89cd-ce3c5256060e None admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:06.366025 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2d3bb-3370-4226-89cd-ce3c5256060e None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:06.366360 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2d3bb-3370-4226-89cd-ce3c5256060e None admin] PATH_INFO: `/v3/projects/93b5040533524222b126bbe93afc56d1/users/ff025cb706d243778468ae4570374690/roles/367179870e5e4f04b19a345f03c6d26b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:06.368677 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2d3bb-3370-4226-89cd-ce3c5256060e None admin] RBAC: Authorizing `identity:create_grant(project_id=93b5040533524222b126bbe93afc56d1, user_id=ff025cb706d243778468ae4570374690, role_id=367179870e5e4f04b19a345f03c6d26b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:06.370129 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2d3bb-3370-4226-89cd-ce3c5256060e None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:06.452104 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 529/1072] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:05:06 2022] PUT /identity/v3/projects/93b5040533524222b126bbe93afc56d1/users/ff025cb706d243778468ae4570374690/roles/367179870e5e4f04b19a345f03c6d26b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:05:06.717785 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414b914-783e-410d-8f46-86c2e8231e34 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:06.718095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414b914-783e-410d-8f46-86c2e8231e34 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:06.718272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414b914-783e-410d-8f46-86c2e8231e34 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:06.719715 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 544/1073] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:06.724739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2fbf1-c64f-4f91-8b1a-6ac3e6b4f358 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:06.725031 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2fbf1-c64f-4f91-8b1a-6ac3e6b4f358 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:06.725307 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2fbf1-c64f-4f91-8b1a-6ac3e6b4f358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:06.742747 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-b6d2fbf1-c64f-4f91-8b1a-6ac3e6b4f358 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:06.743950 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-b6d2fbf1-c64f-4f91-8b1a-6ac3e6b4f358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:06.751566 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:06.810047 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 530/1074] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:05:06.832496 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414b914-783e-410d-8f46-86c2e8231e34 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:06.911923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:06.937059 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414b914-783e-410d-8f46-86c2e8231e34 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:06.972270 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868646b-f536-4c9a-8649-ca6c254368df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:06.972851 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868646b-f536-4c9a-8649-ca6c254368df None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:06.973133 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868646b-f536-4c9a-8649-ca6c254368df None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:06.973406 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868646b-f536-4c9a-8649-ca6c254368df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:07.042935 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3868646b-f536-4c9a-8649-ca6c254368df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:07.044387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3868646b-f536-4c9a-8649-ca6c254368df None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:07.117008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:07.147736 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 545/1075] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:05:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 09:05:07.152331 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2fbf1-c64f-4f91-8b1a-6ac3e6b4f358 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:07.276522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2fbf1-c64f-4f91-8b1a-6ac3e6b4f358 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:07.318034 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21e61d-b70d-4d2c-bf05-f6b912c71946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:07.318837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21e61d-b70d-4d2c-bf05-f6b912c71946 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:07.319157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21e61d-b70d-4d2c-bf05-f6b912c71946 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:07.319432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21e61d-b70d-4d2c-bf05-f6b912c71946 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:07.330025 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21e61d-b70d-4d2c-bf05-f6b912c71946 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:07.331520 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21e61d-b70d-4d2c-bf05-f6b912c71946 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:07.333377 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:07.353478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-cc21e61d-b70d-4d2c-bf05-f6b912c71946 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:05:07.353478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:07.353478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:07.353478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:07.353478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:07.353478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:07.353478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:07.353478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:07.353478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:07.353478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:07.353478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:07.353478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:07.353478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:05:07.353478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:05:07.353478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:05:07.354486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:05:07.354486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:07.354486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:07.354486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:07.354486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:07.354486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:07.354486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:07.354486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:07.354486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:07.354486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:07.354486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:07.354486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:07.354486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:07.354486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:07.354486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:07.354486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:07.354486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:07.355560 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:05:07.355560 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:05:07.355560 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:05:07.355560 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:05:07.361227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 531/1076] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:05:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:07.361855 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868646b-f536-4c9a-8649-ca6c254368df None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:07.466828 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868646b-f536-4c9a-8649-ca6c254368df None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:07.501743 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08665021-1e46-4585-8ff4-3d243fa7a5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:07.502435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08665021-1e46-4585-8ff4-3d243fa7a5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:07.502703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08665021-1e46-4585-8ff4-3d243fa7a5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:07.503002 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08665021-1e46-4585-8ff4-3d243fa7a5c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:07.503731 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08665021-1e46-4585-8ff4-3d243fa7a5c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:07.504038 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08665021-1e46-4585-8ff4-3d243fa7a5c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:07.505426 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08665021-1e46-4585-8ff4-3d243fa7a5c3 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:07.522998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 546/1077] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:07.624509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15568b-109b-4a1a-9f7c-857f24193164 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:07.624814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15568b-109b-4a1a-9f7c-857f24193164 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:07.625064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15568b-109b-4a1a-9f7c-857f24193164 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:07.626791 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 532/1078] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:07.632566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b9d84-a245-4bc7-955a-f585415603d8 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:07.632850 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b9d84-a245-4bc7-955a-f585415603d8 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:07.633136 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b9d84-a245-4bc7-955a-f585415603d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:07.651325 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-5f2b9d84-a245-4bc7-955a-f585415603d8 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:07.652595 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-5f2b9d84-a245-4bc7-955a-f585415603d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:07.682506 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:07.728696 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 547/1079] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:05:07.751366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15568b-109b-4a1a-9f7c-857f24193164 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:07.857762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15568b-109b-4a1a-9f7c-857f24193164 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:07.890353 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157bad3a-8313-440b-9446-9d53b04a0388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:07.891110 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bad3a-8313-440b-9446-9d53b04a0388 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:07.891396 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bad3a-8313-440b-9446-9d53b04a0388 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:07.891669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bad3a-8313-440b-9446-9d53b04a0388 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:07.901351 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157bad3a-8313-440b-9446-9d53b04a0388 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:05:07.909437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-157bad3a-8313-440b-9446-9d53b04a0388 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:05:07.909437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:07.909437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:07.909437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:07.909437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:07.909437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:07.909437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:07.909437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:07.909437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:07.909437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:07.909437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:07.909437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:07.909437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:05:07.909437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:05:07.909437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:05:07.909437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:05:07.910640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:05:07.910640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:05:07.910640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:05:07.910640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:05:07.910640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:07.910640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:07.910640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:07.910640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:07.910640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:07.910640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:07.910640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:07.910640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:07.910640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:07.910640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:07.910640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:07.910640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:07.911643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:07.911643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:07.911643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:07.911643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:07.911643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:05:07.911643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:05:07.911643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:05:07.911643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:05:07.911643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:05:07.911643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:05:07.911643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:05:07.911643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:05:07.911643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 533/1080] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:05:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:07.916739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b9d84-a245-4bc7-955a-f585415603d8 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:08.013060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b9d84-a245-4bc7-955a-f585415603d8 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:08.047068 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf96a84-0c93-468b-9e07-d9a3738cef7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:08.047742 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf96a84-0c93-468b-9e07-d9a3738cef7d None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:08.048035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf96a84-0c93-468b-9e07-d9a3738cef7d None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:08.048319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf96a84-0c93-468b-9e07-d9a3738cef7d None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:08.049073 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf96a84-0c93-468b-9e07-d9a3738cef7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:08.049384 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf96a84-0c93-468b-9e07-d9a3738cef7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:08.050761 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf96a84-0c93-468b-9e07-d9a3738cef7d None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:08.058735 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 548/1081] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:05:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:08.064631 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157bad3a-8313-440b-9446-9d53b04a0388 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:08.158120 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:08.203394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157bad3a-8313-440b-9446-9d53b04a0388 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:08.238001 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc606f-8bbb-4b1d-9e7a-43f978c85e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:08.238871 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc606f-8bbb-4b1d-9e7a-43f978c85e4f None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:08.239188 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc606f-8bbb-4b1d-9e7a-43f978c85e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:08.239470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc606f-8bbb-4b1d-9e7a-43f978c85e4f None admin] PATH_INFO: `/v3/users/896ee122bcd7464e8f0575668b60948e` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:08.240656 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc606f-8bbb-4b1d-9e7a-43f978c85e4f None admin] RBAC: Authorizing `identity:get_user(user_id=896ee122bcd7464e8f0575668b60948e)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:08.242122 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc606f-8bbb-4b1d-9e7a-43f978c85e4f None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:08.243775 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 534/1082] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:05:08 2022] GET /identity/v3/users/896ee122bcd7464e8f0575668b60948e => generated 285 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:08.249335 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf96a84-0c93-468b-9e07-d9a3738cef7d None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:08.276512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:08.354241 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf96a84-0c93-468b-9e07-d9a3738cef7d None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:08.387626 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4669df-6256-45c6-bdef-13939b93ee0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:08.388480 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4669df-6256-45c6-bdef-13939b93ee0b None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:08.388768 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4669df-6256-45c6-bdef-13939b93ee0b None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:08.389046 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4669df-6256-45c6-bdef-13939b93ee0b None admin] PATH_INFO: `/v3/projects/b7d8b91570bb4df0942aa7657b75af24` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:08.390221 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4669df-6256-45c6-bdef-13939b93ee0b None admin] RBAC: Authorizing `identity:get_project(project_id=b7d8b91570bb4df0942aa7657b75af24)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:08.391683 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4669df-6256-45c6-bdef-13939b93ee0b None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:08.393406 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 549/1083] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:05:08 2022] GET /identity/v3/projects/b7d8b91570bb4df0942aa7657b75af24 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:05:08.399232 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc606f-8bbb-4b1d-9e7a-43f978c85e4f None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:08.497456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc606f-8bbb-4b1d-9e7a-43f978c85e4f None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:08.519292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:08.521200 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4b198-7020-4298-880c-02de296546ee None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:08.521490 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4b198-7020-4298-880c-02de296546ee None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:08.521769 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4b198-7020-4298-880c-02de296546ee None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:08.523202 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 550/1084] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:05:08.528967 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b31af4-9e9c-4e4d-acd2-c23b0a904ff0 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:08.529257 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b31af4-9e9c-4e4d-acd2-c23b0a904ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:08.529534 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b31af4-9e9c-4e4d-acd2-c23b0a904ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:08.532711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccda8f8-1a5b-4096-97c2-7bdbd2e8eda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:08.538535 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccda8f8-1a5b-4096-97c2-7bdbd2e8eda8 None admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:08.538535 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccda8f8-1a5b-4096-97c2-7bdbd2e8eda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:08.538535 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccda8f8-1a5b-4096-97c2-7bdbd2e8eda8 None admin] PATH_INFO: `/v3/projects/b7d8b91570bb4df0942aa7657b75af24/users/896ee122bcd7464e8f0575668b60948e/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:08.538535 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccda8f8-1a5b-4096-97c2-7bdbd2e8eda8 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7d8b91570bb4df0942aa7657b75af24, user_id=896ee122bcd7464e8f0575668b60948e, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:08.538535 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccda8f8-1a5b-4096-97c2-7bdbd2e8eda8 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:08.545739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-a8b31af4-9e9c-4e4d-acd2-c23b0a904ff0 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:08.546931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-a8b31af4-9e9c-4e4d-acd2-c23b0a904ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:08.612323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 551/1085] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 09:05:08.628728 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 535/1086] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:05:08 2022] PUT /identity/v3/projects/b7d8b91570bb4df0942aa7657b75af24/users/896ee122bcd7464e8f0575668b60948e/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:05:08.636553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b31af4-9e9c-4e4d-acd2-c23b0a904ff0 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:08.664987 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:08.745672 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b31af4-9e9c-4e4d-acd2-c23b0a904ff0 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:08.786337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d3415-1e28-400f-8761-0eaf657c90a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:08.786337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d3415-1e28-400f-8761-0eaf657c90a6 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:08.786337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d3415-1e28-400f-8761-0eaf657c90a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:08.786337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d3415-1e28-400f-8761-0eaf657c90a6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:08.797829 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d3415-1e28-400f-8761-0eaf657c90a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:05:08.806143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-6f8d3415-1e28-400f-8761-0eaf657c90a6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 09:05:08.806143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:08.806143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:08.806143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:08.806143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:08.806143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:08.806143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:08.806143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:08.806143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:08.806143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:08.806143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:08.806143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:08.806143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:05:08.806143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:05:08.806143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:05:08.806143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:05:08.807390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:05:08.807390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:05:08.807390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:05:08.807390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:05:08.807390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:08.807390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:08.807390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:08.807390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:08.807390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:08.807390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:08.807390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:08.807390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:08.807390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:08.807390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:08.807390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:08.807390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:08.808405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:08.808405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:08.808405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:08.808405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:08.808405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:05:08.808405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:05:08.808405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:05:08.808405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:05:08.808405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:05:08.808405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:05:08.808405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 09:05:08.808405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:05:08.808405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 552/1087] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jun 23 09:05:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:08.813502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccda8f8-1a5b-4096-97c2-7bdbd2e8eda8 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:08.913862 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccda8f8-1a5b-4096-97c2-7bdbd2e8eda8 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:08.948203 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511229a-cc72-496c-9a57-399bb822c92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:08.948901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511229a-cc72-496c-9a57-399bb822c92d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:08.949184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511229a-cc72-496c-9a57-399bb822c92d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:08.949457 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511229a-cc72-496c-9a57-399bb822c92d None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:08.950206 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511229a-cc72-496c-9a57-399bb822c92d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:08.950509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511229a-cc72-496c-9a57-399bb822c92d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:08.952031 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511229a-cc72-496c-9a57-399bb822c92d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:08.960943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 536/1088] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jun 23 09:05:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:08.967141 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d3415-1e28-400f-8761-0eaf657c90a6 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:09.058676 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:09.075037 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d3415-1e28-400f-8761-0eaf657c90a6 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:09.119818 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465cb6a-51f6-429d-bb41-892f35731cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:09.120653 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465cb6a-51f6-429d-bb41-892f35731cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:09.120942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465cb6a-51f6-429d-bb41-892f35731cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:09.121315 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465cb6a-51f6-429d-bb41-892f35731cd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:09.133367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465cb6a-51f6-429d-bb41-892f35731cd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:09.135963 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465cb6a-51f6-429d-bb41-892f35731cd7 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:09.155159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-a465cb6a-51f6-429d-bb41-892f35731cd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:05:09.155159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:09.155159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:09.155159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:09.155159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:09.155159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:09.155159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:09.155159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:09.155159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:09.155159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:09.155159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:09.155159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:09.155159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:05:09.155159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:05:09.155159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:05:09.156249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:05:09.156249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:09.156249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:09.156249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:09.156249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:09.156249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:09.156249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:09.156249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:09.156249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:09.156249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:09.156249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:09.156249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:09.156249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:09.156249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:09.156249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:09.156249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:09.156249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:09.157399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:05:09.157399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:05:09.157399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:05:09.157399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:05:09.159220 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 553/1089] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:09.163726 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511229a-cc72-496c-9a57-399bb822c92d None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:09.221659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:09.278589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511229a-cc72-496c-9a57-399bb822c92d None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:09.311862 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ae4a9-598c-4045-96ba-af17339fbe94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:09.312445 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ae4a9-598c-4045-96ba-af17339fbe94 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:09.312725 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ae4a9-598c-4045-96ba-af17339fbe94 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:09.312998 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ae4a9-598c-4045-96ba-af17339fbe94 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:09.313715 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ae4a9-598c-4045-96ba-af17339fbe94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:09.314072 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ae4a9-598c-4045-96ba-af17339fbe94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:09.315473 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ae4a9-598c-4045-96ba-af17339fbe94 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:09.327982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 537/1090] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:09.333679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465cb6a-51f6-429d-bb41-892f35731cd7 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:09.438645 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465cb6a-51f6-429d-bb41-892f35731cd7 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:09.472670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e24a2-4466-4973-9a63-6d2df82d389f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:09.473465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e24a2-4466-4973-9a63-6d2df82d389f None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:09.473759 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e24a2-4466-4973-9a63-6d2df82d389f None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:09.474036 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e24a2-4466-4973-9a63-6d2df82d389f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:09.476795 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:09.482216 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e24a2-4466-4973-9a63-6d2df82d389f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:09.483711 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e24a2-4466-4973-9a63-6d2df82d389f None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:09.498239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-601e24a2-4466-4973-9a63-6d2df82d389f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 09:05:09.498239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:09.498239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:09.498239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:09.498239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:09.498239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:09.498239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:09.498239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:09.498239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:09.498239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:09.498239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:09.498239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:09.498239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:05:09.498239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:05:09.498239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:05:09.498239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:05:09.499491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:09.499491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:09.499491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:05:09.499491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:05:09.499491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:05:09.499491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:05:09.499491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:09.499491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:09.499491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:09.499491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:09.499491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:09.499491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:09.499491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:09.499491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:09.499491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:09.499491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:09.499491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:09.499491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:09.500521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:09.500521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:09.500521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:05:09.500521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:05:09.500521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:05:09.500521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:05:09.500521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:05:09.500521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:05:09.500521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 09:05:09.500521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:05:09.500521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 554/1091] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jun 23 09:05:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:09.507647 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ae4a9-598c-4045-96ba-af17339fbe94 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:09.614963 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ae4a9-598c-4045-96ba-af17339fbe94 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:09.617265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:09.648956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3aaf79-c467-4844-abc6-d9b49b70ce65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:09.649631 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3aaf79-c467-4844-abc6-d9b49b70ce65 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:09.649922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3aaf79-c467-4844-abc6-d9b49b70ce65 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:09.650199 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3aaf79-c467-4844-abc6-d9b49b70ce65 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:09.651069 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3aaf79-c467-4844-abc6-d9b49b70ce65 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:09.651380 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3aaf79-c467-4844-abc6-d9b49b70ce65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:09.653361 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3aaf79-c467-4844-abc6-d9b49b70ce65 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:09.671518 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 538/1092] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jun 23 09:05:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:09.677168 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e24a2-4466-4973-9a63-6d2df82d389f None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:09.785040 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e24a2-4466-4973-9a63-6d2df82d389f None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:09.822388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85356e-62c8-45d8-ae41-e165d1f56363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:09.823084 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85356e-62c8-45d8-ae41-e165d1f56363 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:09.823386 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85356e-62c8-45d8-ae41-e165d1f56363 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:09.824190 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85356e-62c8-45d8-ae41-e165d1f56363 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:09.883376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:09.890291 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85356e-62c8-45d8-ae41-e165d1f56363 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:09.891777 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85356e-62c8-45d8-ae41-e165d1f56363 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:09.987375 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 555/1093] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:05:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 09:05:09.993168 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3aaf79-c467-4844-abc6-d9b49b70ce65 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:10.091915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3aaf79-c467-4844-abc6-d9b49b70ce65 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:10.129981 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13fdd5-459c-49df-8d69-d02af9671625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:10.130775 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13fdd5-459c-49df-8d69-d02af9671625 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:10.131102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13fdd5-459c-49df-8d69-d02af9671625 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:10.131383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13fdd5-459c-49df-8d69-d02af9671625 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:10.141385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13fdd5-459c-49df-8d69-d02af9671625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:10.142778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13fdd5-459c-49df-8d69-d02af9671625 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:10.160816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-8e13fdd5-459c-49df-8d69-d02af9671625 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:05:10.160816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:10.160816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:10.160816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:10.160816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:10.160816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:10.160816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:10.160816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:10.160816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:10.160816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:10.160816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:10.160816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:10.160816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:05:10.160816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:05:10.160816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:05:10.161824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:05:10.161824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:10.161824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:10.161824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:10.161824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:10.161824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:10.161824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:10.161824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:10.161824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:10.161824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:10.161824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:10.161824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:10.161824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:10.161824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:10.161824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:10.161824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:10.161824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:10.162790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:05:10.162790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:05:10.162790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:05:10.162790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:05:10.162790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 539/1094] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:05:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 09:05:10.168235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85356e-62c8-45d8-ae41-e165d1f56363 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:10.264764 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:10.270595 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85356e-62c8-45d8-ae41-e165d1f56363 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:10.304078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f26dc-468c-4051-9fd9-c96978e4552a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:10.304745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f26dc-468c-4051-9fd9-c96978e4552a None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:10.305035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f26dc-468c-4051-9fd9-c96978e4552a None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:10.305316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f26dc-468c-4051-9fd9-c96978e4552a None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:10.306043 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f26dc-468c-4051-9fd9-c96978e4552a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:10.306353 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f26dc-468c-4051-9fd9-c96978e4552a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:10.307754 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f26dc-468c-4051-9fd9-c96978e4552a None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:10.319321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 556/1095] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:10.325457 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13fdd5-459c-49df-8d69-d02af9671625 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:10.341182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:10.422016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13fdd5-459c-49df-8d69-d02af9671625 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:10.456002 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1e6c1-3bc3-42e6-8f3c-d98f3ff5a429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:10.456993 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1e6c1-3bc3-42e6-8f3c-d98f3ff5a429 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:10.457452 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1e6c1-3bc3-42e6-8f3c-d98f3ff5a429 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:10.457776 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1e6c1-3bc3-42e6-8f3c-d98f3ff5a429 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:10.465757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1e6c1-3bc3-42e6-8f3c-d98f3ff5a429 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:10.467233 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1e6c1-3bc3-42e6-8f3c-d98f3ff5a429 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:10.481161 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-9fb1e6c1-3bc3-42e6-8f3c-d98f3ff5a429 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:05:10.481161 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:10.481161 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:10.481161 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:10.481161 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:10.481161 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:10.481161 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:10.481161 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:10.481161 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:10.481161 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:10.481161 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:10.481161 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:10.481161 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:05:10.481161 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:05:10.481161 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:05:10.482183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:05:10.482183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:10.482183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:10.482183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:10.482183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:10.482183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:10.482183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:10.482183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:10.482183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:10.482183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:10.482183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:10.482183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:10.482183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:10.482183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:10.482183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:10.482183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:10.482183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:10.483208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:05:10.483208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:05:10.483208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:05:10.483208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:05:10.483208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:05:10.483208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:05:10.483208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:05:10.483208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:05:10.483208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 540/1096] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:05:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:10.488187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f26dc-468c-4051-9fd9-c96978e4552a None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:10.584123 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f26dc-468c-4051-9fd9-c96978e4552a None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:10.616732 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13e699-22be-41d1-b252-0f71850c9f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:10.617393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13e699-22be-41d1-b252-0f71850c9f9c None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:10.617676 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13e699-22be-41d1-b252-0f71850c9f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:10.617952 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13e699-22be-41d1-b252-0f71850c9f9c None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:10.618737 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f13e699-22be-41d1-b252-0f71850c9f9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:10.619079 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f13e699-22be-41d1-b252-0f71850c9f9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:10.620552 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f13e699-22be-41d1-b252-0f71850c9f9c None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:10.633189 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 557/1097] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:05:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:10.639017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1e6c1-3bc3-42e6-8f3c-d98f3ff5a429 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:10.661227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db943c9-1cda-476d-8197-059c22f74d6c None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:10.661538 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db943c9-1cda-476d-8197-059c22f74d6c None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:10.661815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db943c9-1cda-476d-8197-059c22f74d6c None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:10.663200 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 558/1098] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:05:10.668605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de6baf-bcd2-439d-8fcd-7fca6a1639af None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:10.668899 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de6baf-bcd2-439d-8fcd-7fca6a1639af None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:10.669176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de6baf-bcd2-439d-8fcd-7fca6a1639af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:10.685824 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-07de6baf-bcd2-439d-8fcd-7fca6a1639af None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:10.687098 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-07de6baf-bcd2-439d-8fcd-7fca6a1639af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:10.697713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:10.715731 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:10.739499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1e6c1-3bc3-42e6-8f3c-d98f3ff5a429 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:10.750529 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 559/1099] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 09:05:10.772654 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1f680-b446-427c-927f-3461da552443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:10.773306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1f680-b446-427c-927f-3461da552443 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:10.773594 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1f680-b446-427c-927f-3461da552443 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:10.773771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de6baf-bcd2-439d-8fcd-7fca6a1639af None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:10.773880 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1f680-b446-427c-927f-3461da552443 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:10.774725 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1f680-b446-427c-927f-3461da552443 None admin] RBAC: Adding query filter params (role.id=367179870e5e4f04b19a345f03c6d26b, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=ff025cb706d243778468ae4570374690) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:10.775071 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1f680-b446-427c-927f-3461da552443 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:10.776552 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1f680-b446-427c-927f-3461da552443 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:10.783056 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 541/1100] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:05:10 2022] GET /identity/v3/role_assignments?user.id=ff025cb706d243778468ae4570374690&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=367179870e5e4f04b19a345f03c6d26b => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:10.884742 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de6baf-bcd2-439d-8fcd-7fca6a1639af None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:10.920316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5188ce-58df-47bb-8304-1912491cb668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:10.921126 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5188ce-58df-47bb-8304-1912491cb668 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:10.921410 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5188ce-58df-47bb-8304-1912491cb668 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:10.921683 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5188ce-58df-47bb-8304-1912491cb668 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:10.932261 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5188ce-58df-47bb-8304-1912491cb668 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:05:10.940834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-9b5188ce-58df-47bb-8304-1912491cb668 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:05:10.940834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:10.940834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:10.940834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:10.940834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:10.940834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:10.940834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:10.940834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:10.940834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:10.940834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:10.940834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:10.940834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:10.940834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:05:10.940834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:05:10.940834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:05:10.940834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:05:10.942034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:05:10.942034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:05:10.942034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:05:10.942034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:05:10.942034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:10.942034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:10.942034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:10.942034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:10.942034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:10.942034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:10.942034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:10.942034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:10.942034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:10.942034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:10.942034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:10.942034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:10.943192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:10.943192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:10.943192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:10.943192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:10.943192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:05:10.943192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:05:10.943192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:05:10.943192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:05:10.943192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:05:10.943192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:05:10.943192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:05:10.943192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:05:10.943192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 560/1101] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:05:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:10.947879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1f680-b446-427c-927f-3461da552443 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:11.052069 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1f680-b446-427c-927f-3461da552443 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:11.086296 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f1463-94e4-47a8-857c-8f0cba5cf501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:11.086996 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f1463-94e4-47a8-857c-8f0cba5cf501 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:11.087282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f1463-94e4-47a8-857c-8f0cba5cf501 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:11.087561 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f1463-94e4-47a8-857c-8f0cba5cf501 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:11.088312 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f1463-94e4-47a8-857c-8f0cba5cf501 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:11.088621 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f1463-94e4-47a8-857c-8f0cba5cf501 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:11.090641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f1463-94e4-47a8-857c-8f0cba5cf501 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:11.095795 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:11.099825 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 542/1102] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:05:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:05:11.106355 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5188ce-58df-47bb-8304-1912491cb668 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:11.116003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:11.210382 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5188ce-58df-47bb-8304-1912491cb668 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:11.246206 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e6b80-a708-4b9b-9975-9d795fc4c6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:11.247126 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e6b80-a708-4b9b-9975-9d795fc4c6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:11.247414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e6b80-a708-4b9b-9975-9d795fc4c6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:11.247692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e6b80-a708-4b9b-9975-9d795fc4c6b5 None admin] PATH_INFO: `/v3/users/896ee122bcd7464e8f0575668b60948e` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:11.248899 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e6b80-a708-4b9b-9975-9d795fc4c6b5 None admin] RBAC: Authorizing `identity:get_user(user_id=896ee122bcd7464e8f0575668b60948e)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:11.250345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e6b80-a708-4b9b-9975-9d795fc4c6b5 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:11.252092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 561/1103] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:05:11 2022] GET /identity/v3/users/896ee122bcd7464e8f0575668b60948e => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:11.258389 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f1463-94e4-47a8-857c-8f0cba5cf501 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:11.359350 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f1463-94e4-47a8-857c-8f0cba5cf501 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:11.393286 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ea558-3075-4050-b778-ff19d3f67d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:11.393939 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ea558-3075-4050-b778-ff19d3f67d42 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:11.394223 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ea558-3075-4050-b778-ff19d3f67d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:11.394496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ea558-3075-4050-b778-ff19d3f67d42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:11.461564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:11.464107 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ea558-3075-4050-b778-ff19d3f67d42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:11.465607 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ea558-3075-4050-b778-ff19d3f67d42 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:11.564130 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 543/1104] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:05:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 09:05:11.570160 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e6b80-a708-4b9b-9975-9d795fc4c6b5 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:11.648588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:11.673231 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e6b80-a708-4b9b-9975-9d795fc4c6b5 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:11.705464 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4587e-3239-4617-9cc7-b5579984bcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:11.706318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4587e-3239-4617-9cc7-b5579984bcbf None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:11.706603 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4587e-3239-4617-9cc7-b5579984bcbf None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:11.706880 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4587e-3239-4617-9cc7-b5579984bcbf None admin] PATH_INFO: `/v3/projects/b7d8b91570bb4df0942aa7657b75af24` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:11.708080 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4587e-3239-4617-9cc7-b5579984bcbf None admin] RBAC: Authorizing `identity:get_project(project_id=b7d8b91570bb4df0942aa7657b75af24)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:11.709517 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4587e-3239-4617-9cc7-b5579984bcbf None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:11.711503 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 562/1105] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:05:11 2022] GET /identity/v3/projects/b7d8b91570bb4df0942aa7657b75af24 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:11.718192 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ea558-3075-4050-b778-ff19d3f67d42 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:11.810884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:11.813730 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ea558-3075-4050-b778-ff19d3f67d42 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:11.846863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b398cd-e3d4-4a62-ba66-8fef55b556ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:11.847550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b398cd-e3d4-4a62-ba66-8fef55b556ba None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:11.847839 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b398cd-e3d4-4a62-ba66-8fef55b556ba None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:11.848121 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b398cd-e3d4-4a62-ba66-8fef55b556ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:11.848946 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b398cd-e3d4-4a62-ba66-8fef55b556ba None admin] RBAC: Adding query filter params (role.id=9cd080b50c6f4ee8a54bf9ef1655bd83, scope.project.id=b7d8b91570bb4df0942aa7657b75af24, user.id=896ee122bcd7464e8f0575668b60948e) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:11.849262 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b398cd-e3d4-4a62-ba66-8fef55b556ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:11.850693 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b398cd-e3d4-4a62-ba66-8fef55b556ba None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:11.856955 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 544/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:05:11 2022] GET /identity/v3/role_assignments?user.id=896ee122bcd7464e8f0575668b60948e&scope.project.id=b7d8b91570bb4df0942aa7657b75af24&role.id=9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:13.890545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc95d81-5f26-4187-9511-1349070fa8c7 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:13.891138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc95d81-5f26-4187-9511-1349070fa8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:13.891138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc95d81-5f26-4187-9511-1349070fa8c7 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:13.892504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 563/1107] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:13.897746 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfb4ad-2cc4-4e54-a298-80e9b4ffd347 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:13.898167 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfb4ad-2cc4-4e54-a298-80e9b4ffd347 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:13.898323 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfb4ad-2cc4-4e54-a298-80e9b4ffd347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:13.915915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-45dfb4ad-2cc4-4e54-a298-80e9b4ffd347 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:13.917101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-45dfb4ad-2cc4-4e54-a298-80e9b4ffd347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:13.982229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 545/1108] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:05:14.006470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc95d81-5f26-4187-9511-1349070fa8c7 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:14.103947 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc95d81-5f26-4187-9511-1349070fa8c7 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:14.136326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6865e9-c831-48d7-9b87-f964e4f99341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:14.137102 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6865e9-c831-48d7-9b87-f964e4f99341 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:14.137388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6865e9-c831-48d7-9b87-f964e4f99341 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:14.137663 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6865e9-c831-48d7-9b87-f964e4f99341 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:14.138815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6865e9-c831-48d7-9b87-f964e4f99341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:14.140259 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6865e9-c831-48d7-9b87-f964e4f99341 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:14.141920 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 564/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:05:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:14.147480 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfb4ad-2cc4-4e54-a298-80e9b4ffd347 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:14.237096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:14.243847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfb4ad-2cc4-4e54-a298-80e9b4ffd347 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:14.277514 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4c2c7-ab6e-4267-a2c6-923b612f53e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:14.278183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4c2c7-ab6e-4267-a2c6-923b612f53e8 None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:14.278466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4c2c7-ab6e-4267-a2c6-923b612f53e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:14.278739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4c2c7-ab6e-4267-a2c6-923b612f53e8 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:14.279495 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4c2c7-ab6e-4267-a2c6-923b612f53e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:14.280936 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4c2c7-ab6e-4267-a2c6-923b612f53e8 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:14.359612 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 546/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jun 23 09:05:14 2022] POST /identity/v3/users => generated 286 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:05:16.420624 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bfd38-d68e-489b-9883-5c22bb529628 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:16.421126 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bfd38-d68e-489b-9883-5c22bb529628 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:16.421126 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bfd38-d68e-489b-9883-5c22bb529628 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:16.422471 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 565/1111] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:16.427772 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c01fc-48a1-4f73-9c8b-0a02cb1441b4 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:16.428156 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c01fc-48a1-4f73-9c8b-0a02cb1441b4 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:16.428335 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c01fc-48a1-4f73-9c8b-0a02cb1441b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:16.452238 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-483c01fc-48a1-4f73-9c8b-0a02cb1441b4 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:16.453416 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-483c01fc-48a1-4f73-9c8b-0a02cb1441b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:16.517750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 547/1112] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:05:16.541584 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bfd38-d68e-489b-9883-5c22bb529628 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:16.553675 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:16.640937 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bfd38-d68e-489b-9883-5c22bb529628 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:16.673454 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c505119-61a7-4d07-902e-2dd8af681f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:16.674237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c505119-61a7-4d07-902e-2dd8af681f60 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:16.674524 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c505119-61a7-4d07-902e-2dd8af681f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:16.674801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c505119-61a7-4d07-902e-2dd8af681f60 None admin] PATH_INFO: `/v3/roles/478b327fd22845cc98a2fafe821b1e85` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:16.676308 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c505119-61a7-4d07-902e-2dd8af681f60 None admin] RBAC: Authorizing `identity:get_role(role_id=478b327fd22845cc98a2fafe821b1e85)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:16.677687 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c505119-61a7-4d07-902e-2dd8af681f60 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:16.679073 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 566/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:05:16 2022] GET /identity/v3/roles/478b327fd22845cc98a2fafe821b1e85 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:16.685227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c01fc-48a1-4f73-9c8b-0a02cb1441b4 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:16.762104 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:16.780154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c01fc-48a1-4f73-9c8b-0a02cb1441b4 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:16.813118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61551e57-efd3-44ae-87fb-4f699acf8d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:16.813981 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61551e57-efd3-44ae-87fb-4f699acf8d4d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:16.814272 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61551e57-efd3-44ae-87fb-4f699acf8d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:16.814553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61551e57-efd3-44ae-87fb-4f699acf8d4d None admin] PATH_INFO: `/v3/users/f908123ee3ae4a899c6553f0589eb1c9` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:16.827557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61551e57-efd3-44ae-87fb-4f699acf8d4d None admin] RBAC: Authorizing `identity:get_user(user_id=f908123ee3ae4a899c6553f0589eb1c9)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:16.829002 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61551e57-efd3-44ae-87fb-4f699acf8d4d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:16.830642 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 548/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:05:16 2022] GET /identity/v3/users/f908123ee3ae4a899c6553f0589eb1c9 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:16.836498 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c505119-61a7-4d07-902e-2dd8af681f60 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:16.911188 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:16.931189 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c505119-61a7-4d07-902e-2dd8af681f60 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:16.964714 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f6e9c-b887-46ae-94e6-0f0b6e932a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:16.965351 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f6e9c-b887-46ae-94e6-0f0b6e932a03 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:16.965639 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f6e9c-b887-46ae-94e6-0f0b6e932a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:16.965920 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f6e9c-b887-46ae-94e6-0f0b6e932a03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:17.030662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f6e9c-b887-46ae-94e6-0f0b6e932a03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:17.032125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f6e9c-b887-46ae-94e6-0f0b6e932a03 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:17.104405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:17.125816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 567/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:05:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 09:05:17.131666 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61551e57-efd3-44ae-87fb-4f699acf8d4d None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:17.228957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61551e57-efd3-44ae-87fb-4f699acf8d4d None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:17.262319 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb8f00-1fc4-4c7f-b45b-9b34c4ae4a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:17.263205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb8f00-1fc4-4c7f-b45b-9b34c4ae4a2e None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:17.263495 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb8f00-1fc4-4c7f-b45b-9b34c4ae4a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:17.263776 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb8f00-1fc4-4c7f-b45b-9b34c4ae4a2e None admin] PATH_INFO: `/v3/projects/b7d8b91570bb4df0942aa7657b75af24` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:17.266483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb8f00-1fc4-4c7f-b45b-9b34c4ae4a2e None admin] RBAC: Authorizing `identity:get_project(project_id=b7d8b91570bb4df0942aa7657b75af24)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:17.268006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb8f00-1fc4-4c7f-b45b-9b34c4ae4a2e None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:17.271691 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 549/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:05:17 2022] GET /identity/v3/projects/b7d8b91570bb4df0942aa7657b75af24 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:17.278289 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f6e9c-b887-46ae-94e6-0f0b6e932a03 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:17.373605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f6e9c-b887-46ae-94e6-0f0b6e932a03 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:17.406600 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c87378-6a03-407c-b284-e230238bfd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:17.407272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c87378-6a03-407c-b284-e230238bfd65 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:17.407562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c87378-6a03-407c-b284-e230238bfd65 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:17.407842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c87378-6a03-407c-b284-e230238bfd65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:17.408663 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c87378-6a03-407c-b284-e230238bfd65 None admin] RBAC: Adding query filter params (role.id=478b327fd22845cc98a2fafe821b1e85, scope.project.id=b7d8b91570bb4df0942aa7657b75af24, user.id=f908123ee3ae4a899c6553f0589eb1c9) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:17.408976 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c87378-6a03-407c-b284-e230238bfd65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:17.410391 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c87378-6a03-407c-b284-e230238bfd65 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:17.415692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 568/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:05:17 2022] GET /identity/v3/role_assignments?user.id=f908123ee3ae4a899c6553f0589eb1c9&scope.project.id=b7d8b91570bb4df0942aa7657b75af24&role.id=478b327fd22845cc98a2fafe821b1e85 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:19.410661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028e3d0-aa5b-48f0-a015-1df478b8555f None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:19.411186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028e3d0-aa5b-48f0-a015-1df478b8555f None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:19.411186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028e3d0-aa5b-48f0-a015-1df478b8555f None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:19.412698 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 550/1118] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:19.417880 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ae8ab-6b24-4293-80f6-89983fd8c1d8 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:19.418272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ae8ab-6b24-4293-80f6-89983fd8c1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:19.418435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ae8ab-6b24-4293-80f6-89983fd8c1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:19.435149 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-720ae8ab-6b24-4293-80f6-89983fd8c1d8 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:19.436312 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-720ae8ab-6b24-4293-80f6-89983fd8c1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:19.469779 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:19.499143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 569/1119] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:05:19.524596 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028e3d0-aa5b-48f0-a015-1df478b8555f None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:19.533904 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:19.624496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028e3d0-aa5b-48f0-a015-1df478b8555f None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:19.658287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718d95c-67a0-445f-ac61-e448d6f77480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:19.659162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718d95c-67a0-445f-ac61-e448d6f77480 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:19.659453 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718d95c-67a0-445f-ac61-e448d6f77480 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:19.659732 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718d95c-67a0-445f-ac61-e448d6f77480 None admin] PATH_INFO: `/v3/roles/478b327fd22845cc98a2fafe821b1e85` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:19.661242 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718d95c-67a0-445f-ac61-e448d6f77480 None admin] RBAC: Authorizing `identity:get_role(role_id=478b327fd22845cc98a2fafe821b1e85)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:19.662573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718d95c-67a0-445f-ac61-e448d6f77480 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:19.663963 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 551/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:05:19 2022] GET /identity/v3/roles/478b327fd22845cc98a2fafe821b1e85 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:19.670332 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ae8ab-6b24-4293-80f6-89983fd8c1d8 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:19.765996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ae8ab-6b24-4293-80f6-89983fd8c1d8 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:19.798664 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40dfa2a-c5f6-4328-b093-3235452be74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:19.799549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40dfa2a-c5f6-4328-b093-3235452be74c None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:19.799839 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40dfa2a-c5f6-4328-b093-3235452be74c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:19.800118 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40dfa2a-c5f6-4328-b093-3235452be74c None admin] PATH_INFO: `/v3/users/f908123ee3ae4a899c6553f0589eb1c9` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:19.801290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40dfa2a-c5f6-4328-b093-3235452be74c None admin] RBAC: Authorizing `identity:get_user(user_id=f908123ee3ae4a899c6553f0589eb1c9)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:19.802722 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40dfa2a-c5f6-4328-b093-3235452be74c None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:19.804354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 570/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:05:19 2022] GET /identity/v3/users/f908123ee3ae4a899c6553f0589eb1c9 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:19.810797 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718d95c-67a0-445f-ac61-e448d6f77480 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:19.882397 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:19.905724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718d95c-67a0-445f-ac61-e448d6f77480 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:19.938854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf956b4-1965-487d-a0b8-c4bd553ba9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:19.939658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf956b4-1965-487d-a0b8-c4bd553ba9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:19.939943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf956b4-1965-487d-a0b8-c4bd553ba9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:19.940217 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf956b4-1965-487d-a0b8-c4bd553ba9b5 None admin] PATH_INFO: `/v3/projects/b7d8b91570bb4df0942aa7657b75af24` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:19.941377 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf956b4-1965-487d-a0b8-c4bd553ba9b5 None admin] RBAC: Authorizing `identity:get_project(project_id=b7d8b91570bb4df0942aa7657b75af24)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:19.942799 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf956b4-1965-487d-a0b8-c4bd553ba9b5 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:19.944485 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 552/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:05:19 2022] GET /identity/v3/projects/b7d8b91570bb4df0942aa7657b75af24 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:19.950325 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40dfa2a-c5f6-4328-b093-3235452be74c None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:19.991094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:20.045353 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40dfa2a-c5f6-4328-b093-3235452be74c None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:20.078598 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65a248-6289-4972-af4a-1b5f55e65c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:20.079516 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65a248-6289-4972-af4a-1b5f55e65c10 None admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:20.079805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65a248-6289-4972-af4a-1b5f55e65c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:20.080088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65a248-6289-4972-af4a-1b5f55e65c10 None admin] PATH_INFO: `/v3/projects/b7d8b91570bb4df0942aa7657b75af24/users/f908123ee3ae4a899c6553f0589eb1c9/roles/478b327fd22845cc98a2fafe821b1e85` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:20.082277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65a248-6289-4972-af4a-1b5f55e65c10 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7d8b91570bb4df0942aa7657b75af24, user_id=f908123ee3ae4a899c6553f0589eb1c9, role_id=478b327fd22845cc98a2fafe821b1e85)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:20.083754 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65a248-6289-4972-af4a-1b5f55e65c10 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:20.163534 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 571/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:05:19 2022] PUT /identity/v3/projects/b7d8b91570bb4df0942aa7657b75af24/users/f908123ee3ae4a899c6553f0589eb1c9/roles/478b327fd22845cc98a2fafe821b1e85 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:05:22.152616 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b3251-9f12-427e-b1e2-b0ff042ab7cd None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:22.153169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b3251-9f12-427e-b1e2-b0ff042ab7cd None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:22.153169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b3251-9f12-427e-b1e2-b0ff042ab7cd None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:22.154539 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 553/1124] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:22.159713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78810c87-72ba-47e8-b181-d452457431cf None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:22.160123 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78810c87-72ba-47e8-b181-d452457431cf None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:22.160283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78810c87-72ba-47e8-b181-d452457431cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:22.177387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-78810c87-72ba-47e8-b181-d452457431cf None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:22.178568 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-78810c87-72ba-47e8-b181-d452457431cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:22.243668 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 572/1125] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:05:22.267675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b3251-9f12-427e-b1e2-b0ff042ab7cd None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:22.366764 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b3251-9f12-427e-b1e2-b0ff042ab7cd None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:22.399798 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db40033-6a16-4b10-940a-34c0ad356f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:22.400637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db40033-6a16-4b10-940a-34c0ad356f74 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:22.400925 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db40033-6a16-4b10-940a-34c0ad356f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:22.401207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db40033-6a16-4b10-940a-34c0ad356f74 None admin] PATH_INFO: `/v3/roles/478b327fd22845cc98a2fafe821b1e85` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:22.403284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db40033-6a16-4b10-940a-34c0ad356f74 None admin] RBAC: Authorizing `identity:get_role(role_id=478b327fd22845cc98a2fafe821b1e85)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:22.404618 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db40033-6a16-4b10-940a-34c0ad356f74 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:22.405963 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 554/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:05:22 2022] GET /identity/v3/roles/478b327fd22845cc98a2fafe821b1e85 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:22.411573 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78810c87-72ba-47e8-b181-d452457431cf None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:22.441207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:22.508390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78810c87-72ba-47e8-b181-d452457431cf None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:22.540823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa50b3-b2cb-4d04-8bfe-8fa3227fec65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:22.541687 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa50b3-b2cb-4d04-8bfe-8fa3227fec65 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:22.541972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa50b3-b2cb-4d04-8bfe-8fa3227fec65 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:22.542764 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa50b3-b2cb-4d04-8bfe-8fa3227fec65 None admin] PATH_INFO: `/v3/users/f908123ee3ae4a899c6553f0589eb1c9` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:22.543988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa50b3-b2cb-4d04-8bfe-8fa3227fec65 None admin] RBAC: Authorizing `identity:get_user(user_id=f908123ee3ae4a899c6553f0589eb1c9)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:22.545425 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa50b3-b2cb-4d04-8bfe-8fa3227fec65 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:22.547081 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 573/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:05:22 2022] GET /identity/v3/users/f908123ee3ae4a899c6553f0589eb1c9 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:22.552983 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db40033-6a16-4b10-940a-34c0ad356f74 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:22.565506 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:22.649317 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db40033-6a16-4b10-940a-34c0ad356f74 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:22.681980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe9fe3-25e0-441f-8b47-28b4b9ee9b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:22.682557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe9fe3-25e0-441f-8b47-28b4b9ee9b9f None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:22.682841 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe9fe3-25e0-441f-8b47-28b4b9ee9b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:22.683690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe9fe3-25e0-441f-8b47-28b4b9ee9b9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:22.747339 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe9fe3-25e0-441f-8b47-28b4b9ee9b9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:22.748775 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe9fe3-25e0-441f-8b47-28b4b9ee9b9f None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:22.759117 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:22.842135 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 555/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:05:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 09:05:22.848055 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa50b3-b2cb-4d04-8bfe-8fa3227fec65 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:22.940824 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:22.944452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa50b3-b2cb-4d04-8bfe-8fa3227fec65 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:22.977090 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611934d-8064-458e-b203-27d18d02d30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:22.977944 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611934d-8064-458e-b203-27d18d02d30a None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:22.978497 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611934d-8064-458e-b203-27d18d02d30a None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:22.978776 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611934d-8064-458e-b203-27d18d02d30a None admin] PATH_INFO: `/v3/projects/b7d8b91570bb4df0942aa7657b75af24` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:22.980032 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611934d-8064-458e-b203-27d18d02d30a None admin] RBAC: Authorizing `identity:get_project(project_id=b7d8b91570bb4df0942aa7657b75af24)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:22.981454 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611934d-8064-458e-b203-27d18d02d30a None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:22.983165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 574/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:05:22 2022] GET /identity/v3/projects/b7d8b91570bb4df0942aa7657b75af24 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:22.989130 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe9fe3-25e0-441f-8b47-28b4b9ee9b9f None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:23.083707 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe9fe3-25e0-441f-8b47-28b4b9ee9b9f None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:23.094183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:23.116001 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7ab8a-7ec0-4f0f-83b2-a007fdb6c77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:23.116641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7ab8a-7ec0-4f0f-83b2-a007fdb6c77d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:23.116926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7ab8a-7ec0-4f0f-83b2-a007fdb6c77d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:23.117203 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7ab8a-7ec0-4f0f-83b2-a007fdb6c77d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:23.118025 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7ab8a-7ec0-4f0f-83b2-a007fdb6c77d None admin] RBAC: Adding query filter params (role.id=478b327fd22845cc98a2fafe821b1e85, scope.project.id=b7d8b91570bb4df0942aa7657b75af24, user.id=f908123ee3ae4a899c6553f0589eb1c9) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:23.118335 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7ab8a-7ec0-4f0f-83b2-a007fdb6c77d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:23.119786 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7ab8a-7ec0-4f0f-83b2-a007fdb6c77d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:23.125885 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 556/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:05:22 2022] GET /identity/v3/role_assignments?user.id=f908123ee3ae4a899c6553f0589eb1c9&scope.project.id=b7d8b91570bb4df0942aa7657b75af24&role.id=478b327fd22845cc98a2fafe821b1e85 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:25.201910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8c6dc-45ec-4ffa-ae4c-f654af7375b1 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:25.202383 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8c6dc-45ec-4ffa-ae4c-f654af7375b1 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:25.202476 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8c6dc-45ec-4ffa-ae4c-f654af7375b1 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:25.203889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 575/1131] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:25.208918 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde908d-4bbe-4632-89d4-363e4f22d2ce None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:25.209351 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde908d-4bbe-4632-89d4-363e4f22d2ce None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:25.209475 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde908d-4bbe-4632-89d4-363e4f22d2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:25.227124 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-4fde908d-4bbe-4632-89d4-363e4f22d2ce None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:25.228332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-4fde908d-4bbe-4632-89d4-363e4f22d2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:25.293396 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 557/1132] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:05:25.316578 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8c6dc-45ec-4ffa-ae4c-f654af7375b1 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:25.413668 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8c6dc-45ec-4ffa-ae4c-f654af7375b1 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:25.445804 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e7961-7237-4fa0-a0cf-969ec6dfe0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:25.446582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e7961-7237-4fa0-a0cf-969ec6dfe0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:25.446868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e7961-7237-4fa0-a0cf-969ec6dfe0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:25.447175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e7961-7237-4fa0-a0cf-969ec6dfe0c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:25.448338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e7961-7237-4fa0-a0cf-969ec6dfe0c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:25.449794 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e7961-7237-4fa0-a0cf-969ec6dfe0c1 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:25.451410 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 576/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:05:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:25.456976 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde908d-4bbe-4632-89d4-363e4f22d2ce None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:25.551216 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde908d-4bbe-4632-89d4-363e4f22d2ce None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:25.553412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:25.583273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c9ae4-c224-43d7-9dc3-7f89a79c126a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:25.583918 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c9ae4-c224-43d7-9dc3-7f89a79c126a None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:25.584202 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c9ae4-c224-43d7-9dc3-7f89a79c126a None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:25.584476 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c9ae4-c224-43d7-9dc3-7f89a79c126a None admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:25.585178 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c9ae4-c224-43d7-9dc3-7f89a79c126a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:25.586857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c9ae4-c224-43d7-9dc3-7f89a79c126a None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:25.703700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 558/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jun 23 09:05:25 2022] POST /identity/v3/projects => generated 311 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:05:27.752983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634b5e2-1c4c-4e0b-bc47-cd32bd53b2d6 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:27.753518 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634b5e2-1c4c-4e0b-bc47-cd32bd53b2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:27.753518 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634b5e2-1c4c-4e0b-bc47-cd32bd53b2d6 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:27.754902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 577/1135] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:27.759932 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ddab6-5dc5-4cd9-8241-d973b2297738 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:27.760318 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ddab6-5dc5-4cd9-8241-d973b2297738 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:27.760496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ddab6-5dc5-4cd9-8241-d973b2297738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:27.777274 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-955ddab6-5dc5-4cd9-8241-d973b2297738 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:27.778445 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-955ddab6-5dc5-4cd9-8241-d973b2297738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:27.843286 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 559/1136] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:05:27.867685 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634b5e2-1c4c-4e0b-bc47-cd32bd53b2d6 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:27.907170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:27.970111 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634b5e2-1c4c-4e0b-bc47-cd32bd53b2d6 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:28.002785 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae717735-229d-4237-905c-694a50a9c3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:28.003636 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae717735-229d-4237-905c-694a50a9c3cb None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:28.003872 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae717735-229d-4237-905c-694a50a9c3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:28.004160 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae717735-229d-4237-905c-694a50a9c3cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:28.005297 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae717735-229d-4237-905c-694a50a9c3cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:28.006719 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae717735-229d-4237-905c-694a50a9c3cb None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:28.008343 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 578/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:05:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:05:28.014360 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ddab6-5dc5-4cd9-8241-d973b2297738 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:28.094334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:28.108968 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ddab6-5dc5-4cd9-8241-d973b2297738 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:28.141088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4346e5-8145-420d-89cd-a49dc242eede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:28.141739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4346e5-8145-420d-89cd-a49dc242eede None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:28.142052 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4346e5-8145-420d-89cd-a49dc242eede None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:28.142329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4346e5-8145-420d-89cd-a49dc242eede None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:28.143072 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4346e5-8145-420d-89cd-a49dc242eede None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:28.144494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4346e5-8145-420d-89cd-a49dc242eede None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:28.422999 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 560/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jun 23 09:05:28 2022] POST /identity/v3/users => generated 286 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:05:30.450720 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fb07e-5892-4c96-8bae-e5e51cdf5b32 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:30.451273 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fb07e-5892-4c96-8bae-e5e51cdf5b32 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:30.451273 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fb07e-5892-4c96-8bae-e5e51cdf5b32 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:30.452577 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 579/1139] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:30.457649 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507e1dc-6f24-4e86-a8b3-d650632022ad None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:30.458017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507e1dc-6f24-4e86-a8b3-d650632022ad None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:30.458221 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507e1dc-6f24-4e86-a8b3-d650632022ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:30.475511 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-a507e1dc-6f24-4e86-a8b3-d650632022ad None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:30.476702 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a507e1dc-6f24-4e86-a8b3-d650632022ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:30.545683 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 561/1140] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:05:30.568797 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1fb07e-5892-4c96-8bae-e5e51cdf5b32 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:30.665652 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1fb07e-5892-4c96-8bae-e5e51cdf5b32 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:30.667182 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:30.702962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb34fa2-44c7-4514-8e62-2a28ee439861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:30.703698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb34fa2-44c7-4514-8e62-2a28ee439861 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:30.703988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb34fa2-44c7-4514-8e62-2a28ee439861 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:30.704271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb34fa2-44c7-4514-8e62-2a28ee439861 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:30.714396 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb34fa2-44c7-4514-8e62-2a28ee439861 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:05:30.722660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-6cb34fa2-44c7-4514-8e62-2a28ee439861 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:05:30.722660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:30.722660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:30.722660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:30.722660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:30.722660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:30.722660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:30.722660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:30.722660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:30.722660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:30.722660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:30.722660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:30.722660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:05:30.722660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:05:30.722660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:05:30.722660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:05:30.723787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:05:30.723787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:05:30.723787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:05:30.723787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:05:30.723787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:30.723787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:30.723787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:30.723787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:30.723787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:30.723787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:30.723787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:30.723787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:30.723787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:30.723787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:30.723787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:30.723787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:30.724883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:30.724883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:30.724883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:30.724883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:30.724883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:05:30.724883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:05:30.724883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:05:30.724883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:05:30.724883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:05:30.724883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:05:30.724883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:05:30.724883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:05:30.724883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 580/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:05:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:30.730700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507e1dc-6f24-4e86-a8b3-d650632022ad None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:30.798565 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:30.826538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507e1dc-6f24-4e86-a8b3-d650632022ad None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:30.859612 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f8b80-1482-4e78-8f28-ca25301e6421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:30.860269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f8b80-1482-4e78-8f28-ca25301e6421 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:30.860556 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f8b80-1482-4e78-8f28-ca25301e6421 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:30.860830 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f8b80-1482-4e78-8f28-ca25301e6421 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:30.861606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f8b80-1482-4e78-8f28-ca25301e6421 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:30.861908 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f8b80-1482-4e78-8f28-ca25301e6421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:30.863301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f8b80-1482-4e78-8f28-ca25301e6421 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:30.872162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 562/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:05:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:30.877966 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb34fa2-44c7-4514-8e62-2a28ee439861 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:30.975926 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb34fa2-44c7-4514-8e62-2a28ee439861 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:31.009200 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad152d60-4287-46a0-aa1a-b4881858ba39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:31.010109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad152d60-4287-46a0-aa1a-b4881858ba39 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:31.010523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad152d60-4287-46a0-aa1a-b4881858ba39 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:31.011093 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad152d60-4287-46a0-aa1a-b4881858ba39 None admin] PATH_INFO: `/v3/users/c23627cd385a4c49a525dd8f433c5bd5` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:31.026647 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad152d60-4287-46a0-aa1a-b4881858ba39 None admin] RBAC: Authorizing `identity:get_user(user_id=c23627cd385a4c49a525dd8f433c5bd5)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:31.028129 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad152d60-4287-46a0-aa1a-b4881858ba39 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:31.029831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 581/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:05:30 2022] GET /identity/v3/users/c23627cd385a4c49a525dd8f433c5bd5 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:31.035609 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f8b80-1482-4e78-8f28-ca25301e6421 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:31.134734 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f8b80-1482-4e78-8f28-ca25301e6421 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:31.168240 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8982845d-540f-4869-9571-ba56a9d79fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:31.168892 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982845d-540f-4869-9571-ba56a9d79fdb None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:31.169183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982845d-540f-4869-9571-ba56a9d79fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:31.169466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982845d-540f-4869-9571-ba56a9d79fdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:31.172161 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:31.235994 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8982845d-540f-4869-9571-ba56a9d79fdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:31.237385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8982845d-540f-4869-9571-ba56a9d79fdb None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:31.352514 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 563/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:05:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 09:05:31.360749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad152d60-4287-46a0-aa1a-b4881858ba39 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:31.387304 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:31.461816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad152d60-4287-46a0-aa1a-b4881858ba39 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:31.495447 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a38466-ef78-40eb-b2d7-27462a2507c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:31.496242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a38466-ef78-40eb-b2d7-27462a2507c8 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:31.496528 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a38466-ef78-40eb-b2d7-27462a2507c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:31.496808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a38466-ef78-40eb-b2d7-27462a2507c8 None admin] PATH_INFO: `/v3/projects/5978a24db7dd4549be8ca305afa25714` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:31.497987 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a38466-ef78-40eb-b2d7-27462a2507c8 None admin] RBAC: Authorizing `identity:get_project(project_id=5978a24db7dd4549be8ca305afa25714)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:31.499443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a38466-ef78-40eb-b2d7-27462a2507c8 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:31.501127 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 582/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:05:31 2022] GET /identity/v3/projects/5978a24db7dd4549be8ca305afa25714 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:31.507202 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8982845d-540f-4869-9571-ba56a9d79fdb None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:31.540725 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:31.605214 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8982845d-540f-4869-9571-ba56a9d79fdb None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:31.637444 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd41388-e9f5-47ae-ba53-7c8ad601001b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:31.638080 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd41388-e9f5-47ae-ba53-7c8ad601001b None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:31.638364 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd41388-e9f5-47ae-ba53-7c8ad601001b None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:31.638639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd41388-e9f5-47ae-ba53-7c8ad601001b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:31.639484 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd41388-e9f5-47ae-ba53-7c8ad601001b None admin] RBAC: Adding query filter params (role.id=9cd080b50c6f4ee8a54bf9ef1655bd83, scope.project.id=5978a24db7dd4549be8ca305afa25714, user.id=c23627cd385a4c49a525dd8f433c5bd5) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:31.639794 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd41388-e9f5-47ae-ba53-7c8ad601001b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:31.641217 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd41388-e9f5-47ae-ba53-7c8ad601001b None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:31.646444 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 564/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:05:31 2022] GET /identity/v3/role_assignments?user.id=c23627cd385a4c49a525dd8f433c5bd5&scope.project.id=5978a24db7dd4549be8ca305afa25714&role.id=9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:33.677092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8b1f2-e13b-4f3f-9240-9a1d9db3f1a5 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:33.677660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8b1f2-e13b-4f3f-9240-9a1d9db3f1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:33.678049 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8b1f2-e13b-4f3f-9240-9a1d9db3f1a5 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:33.680528 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 583/1147] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:33.686746 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b098c-34f7-427e-b640-3702a1ad8f2d None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:33.687222 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b098c-34f7-427e-b640-3702a1ad8f2d None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:33.687339 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b098c-34f7-427e-b640-3702a1ad8f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:33.707465 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-0e8b098c-34f7-427e-b640-3702a1ad8f2d None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:33.708660 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-0e8b098c-34f7-427e-b640-3702a1ad8f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:33.773056 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 565/1148] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:05:33.796240 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8b1f2-e13b-4f3f-9240-9a1d9db3f1a5 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:33.881985 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:33.894818 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8b1f2-e13b-4f3f-9240-9a1d9db3f1a5 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:33.927490 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7cb931-9975-4970-b262-ee379e9565c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:33.928213 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cb931-9975-4970-b262-ee379e9565c8 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:33.928497 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cb931-9975-4970-b262-ee379e9565c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:33.928774 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cb931-9975-4970-b262-ee379e9565c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:33.938502 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7cb931-9975-4970-b262-ee379e9565c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:05:33.946094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-2e7cb931-9975-4970-b262-ee379e9565c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:05:33.946094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:33.946094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:33.946094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:33.946094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:33.946094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:33.946094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:33.946094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:33.946094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:33.946094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:33.946094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:33.946094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:33.946094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:05:33.946094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:05:33.946094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:05:33.946094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:05:33.947298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:05:33.947298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:05:33.947298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:05:33.947298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:05:33.947298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:33.947298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:33.947298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:33.947298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:33.947298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:33.947298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:33.947298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:33.947298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:33.947298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:33.947298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:33.947298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:33.947298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:33.949048 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:33.949048 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:33.949048 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:33.949048 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:33.949048 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:05:33.949048 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:05:33.949048 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:05:33.949048 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:05:33.949048 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:05:33.949048 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:05:33.949048 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:05:33.949048 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:05:33.949048 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 584/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:05:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:33.953891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b098c-34f7-427e-b640-3702a1ad8f2d None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:33.981593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:34.049299 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b098c-34f7-427e-b640-3702a1ad8f2d None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:34.081602 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f625b0-0b86-41fc-aec9-01351288d65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:34.082252 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f625b0-0b86-41fc-aec9-01351288d65e None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:34.082535 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f625b0-0b86-41fc-aec9-01351288d65e None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:34.082809 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f625b0-0b86-41fc-aec9-01351288d65e None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:34.083583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f625b0-0b86-41fc-aec9-01351288d65e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:34.083889 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f625b0-0b86-41fc-aec9-01351288d65e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:34.085255 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f625b0-0b86-41fc-aec9-01351288d65e None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:34.093322 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 566/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:05:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:34.098960 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7cb931-9975-4970-b262-ee379e9565c8 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:34.195918 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7cb931-9975-4970-b262-ee379e9565c8 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:34.228302 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140cdf3-2dcf-4a8d-b156-d96be25d3cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:34.229165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140cdf3-2dcf-4a8d-b156-d96be25d3cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:34.229452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140cdf3-2dcf-4a8d-b156-d96be25d3cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:34.230310 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140cdf3-2dcf-4a8d-b156-d96be25d3cb2 None admin] PATH_INFO: `/v3/users/c23627cd385a4c49a525dd8f433c5bd5` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:34.231521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140cdf3-2dcf-4a8d-b156-d96be25d3cb2 None admin] RBAC: Authorizing `identity:get_user(user_id=c23627cd385a4c49a525dd8f433c5bd5)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:34.232955 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140cdf3-2dcf-4a8d-b156-d96be25d3cb2 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:34.234572 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 585/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:05:34 2022] GET /identity/v3/users/c23627cd385a4c49a525dd8f433c5bd5 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:34.240313 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f625b0-0b86-41fc-aec9-01351288d65e None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:34.329246 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:34.338661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f625b0-0b86-41fc-aec9-01351288d65e None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:34.370231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6986d9-acd5-4380-a8e5-aab34b2294de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:34.371620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6986d9-acd5-4380-a8e5-aab34b2294de None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:34.371909 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6986d9-acd5-4380-a8e5-aab34b2294de None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:34.372187 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6986d9-acd5-4380-a8e5-aab34b2294de None admin] PATH_INFO: `/v3/projects/5978a24db7dd4549be8ca305afa25714` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:34.373364 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6986d9-acd5-4380-a8e5-aab34b2294de None admin] RBAC: Authorizing `identity:get_project(project_id=5978a24db7dd4549be8ca305afa25714)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:34.374788 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6986d9-acd5-4380-a8e5-aab34b2294de None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:34.376500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 567/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:05:34 2022] GET /identity/v3/projects/5978a24db7dd4549be8ca305afa25714 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:34.382391 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140cdf3-2dcf-4a8d-b156-d96be25d3cb2 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:34.401131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:34.479937 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140cdf3-2dcf-4a8d-b156-d96be25d3cb2 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:34.512045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d5531-336a-4ed1-b2bb-6cf03a4bb8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:34.512906 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d5531-336a-4ed1-b2bb-6cf03a4bb8d7 None admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:34.513188 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d5531-336a-4ed1-b2bb-6cf03a4bb8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:34.513461 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d5531-336a-4ed1-b2bb-6cf03a4bb8d7 None admin] PATH_INFO: `/v3/projects/5978a24db7dd4549be8ca305afa25714/users/c23627cd385a4c49a525dd8f433c5bd5/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:34.515418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d5531-336a-4ed1-b2bb-6cf03a4bb8d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5978a24db7dd4549be8ca305afa25714, user_id=c23627cd385a4c49a525dd8f433c5bd5, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:34.516831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d5531-336a-4ed1-b2bb-6cf03a4bb8d7 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:34.639714 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 586/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:05:34 2022] PUT /identity/v3/projects/5978a24db7dd4549be8ca305afa25714/users/c23627cd385a4c49a525dd8f433c5bd5/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:05:36.627597 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26186ab-d7f0-4bad-ac74-88b36fa3923e None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:36.628154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26186ab-d7f0-4bad-ac74-88b36fa3923e None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:36.628154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26186ab-d7f0-4bad-ac74-88b36fa3923e None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:36.629518 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 568/1154] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:36.634836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53702c42-2495-4a1a-8281-97496b568cff None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:36.635235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53702c42-2495-4a1a-8281-97496b568cff None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:36.635448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53702c42-2495-4a1a-8281-97496b568cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:36.652235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-53702c42-2495-4a1a-8281-97496b568cff None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:36.653408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-53702c42-2495-4a1a-8281-97496b568cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:36.722001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 587/1155] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:05:36.746045 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26186ab-d7f0-4bad-ac74-88b36fa3923e None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:36.843235 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26186ab-d7f0-4bad-ac74-88b36fa3923e None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:36.875598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f8f7d-06eb-442e-945d-05518559bb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:36.876422 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f8f7d-06eb-442e-945d-05518559bb38 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:36.876741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f8f7d-06eb-442e-945d-05518559bb38 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:36.877018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f8f7d-06eb-442e-945d-05518559bb38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:36.887761 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f8f7d-06eb-442e-945d-05518559bb38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:05:36.895231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-3e9f8f7d-06eb-442e-945d-05518559bb38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:05:36.895231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:36.895231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:36.895231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:36.895231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:36.895231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:36.895231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:36.895231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:36.895231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:36.895231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:36.895231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:36.895231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:36.895231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:05:36.895231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:05:36.895231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:05:36.895231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:05:36.896557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:05:36.896557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:05:36.896557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:05:36.896557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:05:36.896557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:36.896557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:36.896557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:36.896557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:36.896557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:36.896557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:36.896557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:36.896557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:36.896557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:36.896557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:36.896557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:36.896557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:36.897760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:36.897760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:36.897760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:36.897760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:36.897760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:05:36.897760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:05:36.897760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:05:36.897760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:05:36.897760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:05:36.897760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:05:36.897760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:05:36.897760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:05:36.897760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 569/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:05:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:36.903170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53702c42-2495-4a1a-8281-97496b568cff None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:36.911366 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:36.997107 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53702c42-2495-4a1a-8281-97496b568cff None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:37.030582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1897ce5-b4c1-4d21-b3d1-8e5bf56be52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:37.031261 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1897ce5-b4c1-4d21-b3d1-8e5bf56be52b None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:37.031549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1897ce5-b4c1-4d21-b3d1-8e5bf56be52b None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:37.031828 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1897ce5-b4c1-4d21-b3d1-8e5bf56be52b None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:37.032568 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1897ce5-b4c1-4d21-b3d1-8e5bf56be52b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:37.032874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1897ce5-b4c1-4d21-b3d1-8e5bf56be52b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:37.034229 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1897ce5-b4c1-4d21-b3d1-8e5bf56be52b None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:37.042037 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 588/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:05:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:05:37.047611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f8f7d-06eb-442e-945d-05518559bb38 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:37.065733 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:37.145510 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f8f7d-06eb-442e-945d-05518559bb38 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:37.178723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526dc83-22e7-47f8-9c2f-b773ecaa3c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:37.179623 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526dc83-22e7-47f8-9c2f-b773ecaa3c0b None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:37.179911 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526dc83-22e7-47f8-9c2f-b773ecaa3c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:37.180191 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526dc83-22e7-47f8-9c2f-b773ecaa3c0b None admin] PATH_INFO: `/v3/users/c23627cd385a4c49a525dd8f433c5bd5` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:37.181376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526dc83-22e7-47f8-9c2f-b773ecaa3c0b None admin] RBAC: Authorizing `identity:get_user(user_id=c23627cd385a4c49a525dd8f433c5bd5)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:37.183341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526dc83-22e7-47f8-9c2f-b773ecaa3c0b None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:37.184967 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 570/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:05:37 2022] GET /identity/v3/users/c23627cd385a4c49a525dd8f433c5bd5 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:37.190546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1897ce5-b4c1-4d21-b3d1-8e5bf56be52b None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:37.255733 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:37.288544 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1897ce5-b4c1-4d21-b3d1-8e5bf56be52b None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:37.321832 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01e268-5259-4b37-b24c-c2a6e7bb19e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:37.322474 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01e268-5259-4b37-b24c-c2a6e7bb19e5 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:37.322765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01e268-5259-4b37-b24c-c2a6e7bb19e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:37.323074 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01e268-5259-4b37-b24c-c2a6e7bb19e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:37.387369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01e268-5259-4b37-b24c-c2a6e7bb19e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:37.388810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01e268-5259-4b37-b24c-c2a6e7bb19e5 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:37.453619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:37.486074 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 589/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:05:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 09:05:37.491889 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526dc83-22e7-47f8-9c2f-b773ecaa3c0b None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:37.572434 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:37.586212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526dc83-22e7-47f8-9c2f-b773ecaa3c0b None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:37.618414 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c0445-3065-437b-b757-03c746da302a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:37.619282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c0445-3065-437b-b757-03c746da302a None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:37.619568 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c0445-3065-437b-b757-03c746da302a None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:37.619842 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c0445-3065-437b-b757-03c746da302a None admin] PATH_INFO: `/v3/projects/5978a24db7dd4549be8ca305afa25714` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:37.621524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c0445-3065-437b-b757-03c746da302a None admin] RBAC: Authorizing `identity:get_project(project_id=5978a24db7dd4549be8ca305afa25714)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:37.622977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c0445-3065-437b-b757-03c746da302a None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:37.624666 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 571/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:05:37 2022] GET /identity/v3/projects/5978a24db7dd4549be8ca305afa25714 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:37.630621 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01e268-5259-4b37-b24c-c2a6e7bb19e5 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:37.724810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01e268-5259-4b37-b24c-c2a6e7bb19e5 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:37.756749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52f50e-25d0-4a3c-b70f-baa42f5ee1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:37.757393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52f50e-25d0-4a3c-b70f-baa42f5ee1e1 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:37.757679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52f50e-25d0-4a3c-b70f-baa42f5ee1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:37.757956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52f50e-25d0-4a3c-b70f-baa42f5ee1e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:37.758787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52f50e-25d0-4a3c-b70f-baa42f5ee1e1 None admin] RBAC: Adding query filter params (role.id=9cd080b50c6f4ee8a54bf9ef1655bd83, scope.project.id=5978a24db7dd4549be8ca305afa25714, user.id=c23627cd385a4c49a525dd8f433c5bd5) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:37.759126 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52f50e-25d0-4a3c-b70f-baa42f5ee1e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:37.760544 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52f50e-25d0-4a3c-b70f-baa42f5ee1e1 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:37.766661 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 590/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:05:37 2022] GET /identity/v3/role_assignments?user.id=c23627cd385a4c49a525dd8f433c5bd5&scope.project.id=5978a24db7dd4549be8ca305afa25714&role.id=9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:39.796038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc25a7-9aa8-4ce2-93ba-d2c08d8a2a78 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:39.796605 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc25a7-9aa8-4ce2-93ba-d2c08d8a2a78 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:39.796605 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc25a7-9aa8-4ce2-93ba-d2c08d8a2a78 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:39.798015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 572/1162] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:39.803192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03240b0-3269-42bd-b221-bfcd4956e82a None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:39.803576 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03240b0-3269-42bd-b221-bfcd4956e82a None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:39.803757 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03240b0-3269-42bd-b221-bfcd4956e82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:39.820552 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-f03240b0-3269-42bd-b221-bfcd4956e82a None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:39.821730 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-f03240b0-3269-42bd-b221-bfcd4956e82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:39.847582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:39.886545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 591/1163] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:05:39.908759 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc25a7-9aa8-4ce2-93ba-d2c08d8a2a78 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:40.007053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc25a7-9aa8-4ce2-93ba-d2c08d8a2a78 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:40.059880 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b18c02-6510-4dc9-8350-28f9bb76a9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:40.060458 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b18c02-6510-4dc9-8350-28f9bb76a9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:40.060744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b18c02-6510-4dc9-8350-28f9bb76a9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:40.061023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b18c02-6510-4dc9-8350-28f9bb76a9e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:40.084218 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:40.126390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b18c02-6510-4dc9-8350-28f9bb76a9e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:40.127851 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b18c02-6510-4dc9-8350-28f9bb76a9e0 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:40.222828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 573/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:05:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 09:05:40.228778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03240b0-3269-42bd-b221-bfcd4956e82a None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:40.323656 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03240b0-3269-42bd-b221-bfcd4956e82a None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:40.357962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887aa4a-2478-45ae-8a9c-8d9727d7cfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:40.358755 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887aa4a-2478-45ae-8a9c-8d9727d7cfe6 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:40.359075 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887aa4a-2478-45ae-8a9c-8d9727d7cfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:40.359358 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887aa4a-2478-45ae-8a9c-8d9727d7cfe6 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:40.366070 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5887aa4a-2478-45ae-8a9c-8d9727d7cfe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:40.367549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5887aa4a-2478-45ae-8a9c-8d9727d7cfe6 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:40.381510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-5887aa4a-2478-45ae-8a9c-8d9727d7cfe6 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 09:05:40.381510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:40.381510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 23 09:05:40.381510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 23 09:05:40.381510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:05:40.381510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:05:40.381510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:05:40.381510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:05:40.381510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 23 09:05:40.381510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:05:40.381510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:05:40.381510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:05:40.381510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:40.381510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:40.381510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:40.381510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:40.381510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:40.382846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:40.382846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:40.382846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:40.382846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:40.382846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:40.382846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:40.382846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:05:40.382846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:05:40.382846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:05:40.382846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:05:40.382846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:40.382846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:40.382846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:40.382846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:40.382846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:40.382846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:40.383827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:40.383827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:40.383827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:40.383827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:40.383827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:40.383827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:40.383827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:40.383827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:40.383827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:40.383827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:40.383827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 23 09:05:40.383827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:05:40.383827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 09:05:40.383827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:05:40.383827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 592/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jun 23 09:05:40 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:40.389642 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b18c02-6510-4dc9-8350-28f9bb76a9e0 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:40.446273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:40.488231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b18c02-6510-4dc9-8350-28f9bb76a9e0 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:40.520633 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66bfe4b-5c1f-4ce8-b87b-a929f4f2e791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:40.521277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bfe4b-5c1f-4ce8-b87b-a929f4f2e791 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:40.521560 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bfe4b-5c1f-4ce8-b87b-a929f4f2e791 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:40.521833 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bfe4b-5c1f-4ce8-b87b-a929f4f2e791 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:40.522553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bfe4b-5c1f-4ce8-b87b-a929f4f2e791 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:40.522856 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bfe4b-5c1f-4ce8-b87b-a929f4f2e791 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:40.524240 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bfe4b-5c1f-4ce8-b87b-a929f4f2e791 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:40.531087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 574/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jun 23 09:05:40 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:42.527134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba685d10-03c5-447c-b131-6a9644541117 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:42.527649 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba685d10-03c5-447c-b131-6a9644541117 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:42.527649 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba685d10-03c5-447c-b131-6a9644541117 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:42.528998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 593/1167] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:42.534206 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558b5c7-a02f-428d-ad7b-7e3cea64ed0d None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:42.534597 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558b5c7-a02f-428d-ad7b-7e3cea64ed0d None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:42.534771 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558b5c7-a02f-428d-ad7b-7e3cea64ed0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:42.551505 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-0558b5c7-a02f-428d-ad7b-7e3cea64ed0d None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:42.552667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-0558b5c7-a02f-428d-ad7b-7e3cea64ed0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:42.616346 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 575/1168] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:05:42.638878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba685d10-03c5-447c-b131-6a9644541117 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:42.663614 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:42.739571 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba685d10-03c5-447c-b131-6a9644541117 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:42.772041 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f3b0a-b8a5-4660-aed4-472e90b5a4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:42.772690 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f3b0a-b8a5-4660-aed4-472e90b5a4bc None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:42.772977 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f3b0a-b8a5-4660-aed4-472e90b5a4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:42.773250 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f3b0a-b8a5-4660-aed4-472e90b5a4bc None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:42.773994 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f3b0a-b8a5-4660-aed4-472e90b5a4bc None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:42.775392 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f3b0a-b8a5-4660-aed4-472e90b5a4bc None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:42.862994 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 594/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Thu Jun 23 09:05:42 2022] POST /identity/v3/domains => generated 267 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:05:44.878842 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0de81-63be-436c-a80a-6486d9908329 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:44.879400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0de81-63be-436c-a80a-6486d9908329 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:44.879498 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0de81-63be-436c-a80a-6486d9908329 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:44.880805 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 576/1170] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:44.886507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f1efa-655e-44bd-b706-964f28ff40f4 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:44.886882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f1efa-655e-44bd-b706-964f28ff40f4 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:44.887114 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f1efa-655e-44bd-b706-964f28ff40f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:44.903921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-489f1efa-655e-44bd-b706-964f28ff40f4 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:44.905091 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-489f1efa-655e-44bd-b706-964f28ff40f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:44.968740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 595/1171] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:05:44.992423 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0de81-63be-436c-a80a-6486d9908329 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:45.041340 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:45.091273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0de81-63be-436c-a80a-6486d9908329 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:45.124155 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3d1dd-d8a9-4de5-a874-350daf77d2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:45.125009 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3d1dd-d8a9-4de5-a874-350daf77d2c0 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:45.125300 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3d1dd-d8a9-4de5-a874-350daf77d2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:45.125582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3d1dd-d8a9-4de5-a874-350daf77d2c0 None admin] PATH_INFO: `/v3/domains/d1eed68980404b3dad0f9a5beb3febb1` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:45.136267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3d1dd-d8a9-4de5-a874-350daf77d2c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d1eed68980404b3dad0f9a5beb3febb1)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:45.137690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3d1dd-d8a9-4de5-a874-350daf77d2c0 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:45.139324 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 577/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Thu Jun 23 09:05:44 2022] GET /identity/v3/domains/d1eed68980404b3dad0f9a5beb3febb1 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:45.145099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f1efa-655e-44bd-b706-964f28ff40f4 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:45.159648 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:45.241362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f1efa-655e-44bd-b706-964f28ff40f4 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:45.273638 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b67a6-97fb-4979-8296-bb601c4c3c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:45.274297 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b67a6-97fb-4979-8296-bb601c4c3c73 None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:45.274584 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b67a6-97fb-4979-8296-bb601c4c3c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:45.274863 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b67a6-97fb-4979-8296-bb601c4c3c73 None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:45.275611 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b67a6-97fb-4979-8296-bb601c4c3c73 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:45.277634 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b67a6-97fb-4979-8296-bb601c4c3c73 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:45.405653 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 596/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jun 23 09:05:45 2022] POST /identity/v3/projects => generated 361 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:05:47.487812 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33edd9-ccb6-4ca8-9158-a85564c89542 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:47.487812 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33edd9-ccb6-4ca8-9158-a85564c89542 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:47.487812 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33edd9-ccb6-4ca8-9158-a85564c89542 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:47.489190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 578/1174] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:47.494531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ae1b6-4f4f-4ee9-b278-58e793451e2e None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:47.494975 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ae1b6-4f4f-4ee9-b278-58e793451e2e None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:47.495146 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ae1b6-4f4f-4ee9-b278-58e793451e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:47.512064 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-551ae1b6-4f4f-4ee9-b278-58e793451e2e None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:47.513247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-551ae1b6-4f4f-4ee9-b278-58e793451e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:47.569329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:47.577599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 597/1175] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 09:05:47.602926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33edd9-ccb6-4ca8-9158-a85564c89542 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:47.702027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33edd9-ccb6-4ca8-9158-a85564c89542 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:47.710552 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:47.734428 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210ad76-d098-48a3-bc81-41f7710ee149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:47.735312 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210ad76-d098-48a3-bc81-41f7710ee149 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:47.735601 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210ad76-d098-48a3-bc81-41f7710ee149 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:47.735879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210ad76-d098-48a3-bc81-41f7710ee149 None admin] PATH_INFO: `/v3/domains/d1eed68980404b3dad0f9a5beb3febb1` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:47.737053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d210ad76-d098-48a3-bc81-41f7710ee149 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d1eed68980404b3dad0f9a5beb3febb1)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:47.738544 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d210ad76-d098-48a3-bc81-41f7710ee149 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:47.740147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 579/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Thu Jun 23 09:05:47 2022] GET /identity/v3/domains/d1eed68980404b3dad0f9a5beb3febb1 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:47.746372 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ae1b6-4f4f-4ee9-b278-58e793451e2e None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:47.840348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ae1b6-4f4f-4ee9-b278-58e793451e2e None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:47.872391 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d89b7c-b820-43b7-8476-2e3f9e082abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:47.873055 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d89b7c-b820-43b7-8476-2e3f9e082abc None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:47.873341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d89b7c-b820-43b7-8476-2e3f9e082abc None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:47.873615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d89b7c-b820-43b7-8476-2e3f9e082abc None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:47.874315 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d89b7c-b820-43b7-8476-2e3f9e082abc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:47.875762 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d89b7c-b820-43b7-8476-2e3f9e082abc None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:47.969948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 598/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jun 23 09:05:47 2022] POST /identity/v3/users => generated 311 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:05:50.033703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e517869-6b12-44d7-b170-e91dc32303af None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:50.034254 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e517869-6b12-44d7-b170-e91dc32303af None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:50.034254 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e517869-6b12-44d7-b170-e91dc32303af None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:50.035682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 580/1178] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:50.040740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa410a4b-587b-48c3-addc-8fe9abb04770 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:50.041114 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa410a4b-587b-48c3-addc-8fe9abb04770 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:50.041306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa410a4b-587b-48c3-addc-8fe9abb04770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:50.058258 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-fa410a4b-587b-48c3-addc-8fe9abb04770 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:50.059478 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-fa410a4b-587b-48c3-addc-8fe9abb04770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:50.103823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:50.127080 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 599/1179] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:05:50.147159 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e517869-6b12-44d7-b170-e91dc32303af None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:50.246629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e517869-6b12-44d7-b170-e91dc32303af None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:50.279225 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5154fb-80c7-42b7-affb-ccdefa7ae4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:50.280004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5154fb-80c7-42b7-affb-ccdefa7ae4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:50.280288 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5154fb-80c7-42b7-affb-ccdefa7ae4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:50.280563 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5154fb-80c7-42b7-affb-ccdefa7ae4c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:50.290757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5154fb-80c7-42b7-affb-ccdefa7ae4c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:05:50.298212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-6f5154fb-80c7-42b7-affb-ccdefa7ae4c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:05:50.298212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:50.298212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:50.298212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:50.298212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:50.298212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:50.298212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:50.298212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:50.298212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:50.298212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:50.298212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:50.298212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:50.298212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:05:50.298212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:05:50.298212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:05:50.298212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:05:50.299437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:05:50.299437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:05:50.299437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:05:50.299437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:05:50.299437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:50.299437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:50.299437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:50.299437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:50.299437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:50.299437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:50.299437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:50.299437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:50.299437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:50.299437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:50.299437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:50.299437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:50.300696 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:50.300696 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:50.300696 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:50.300696 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:50.300696 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:05:50.300696 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:05:50.300696 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:05:50.300696 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:05:50.300696 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:05:50.300696 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:05:50.300696 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:05:50.300696 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:05:50.300696 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 581/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:05:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:50.305520 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa410a4b-587b-48c3-addc-8fe9abb04770 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:50.405179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa410a4b-587b-48c3-addc-8fe9abb04770 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:50.437519 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033671f-0996-4cce-9f91-d9fbde82794b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:50.438187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033671f-0996-4cce-9f91-d9fbde82794b None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:50.438474 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033671f-0996-4cce-9f91-d9fbde82794b None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:50.438753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033671f-0996-4cce-9f91-d9fbde82794b None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:50.439529 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033671f-0996-4cce-9f91-d9fbde82794b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:50.439841 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033671f-0996-4cce-9f91-d9fbde82794b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:50.441208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033671f-0996-4cce-9f91-d9fbde82794b None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:50.449733 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 600/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:05:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:50.455554 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5154fb-80c7-42b7-affb-ccdefa7ae4c4 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:50.489485 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:50.552983 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5154fb-80c7-42b7-affb-ccdefa7ae4c4 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:50.585507 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf2752-5d54-4df8-991b-6d7a568f76a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:50.586618 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf2752-5d54-4df8-991b-6d7a568f76a3 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:50.586897 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf2752-5d54-4df8-991b-6d7a568f76a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:50.587199 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf2752-5d54-4df8-991b-6d7a568f76a3 None admin] PATH_INFO: `/v3/users/860d90a31aad429c8460c1f0baf87073` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:50.600417 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf2752-5d54-4df8-991b-6d7a568f76a3 None admin] RBAC: Authorizing `identity:get_user(user_id=860d90a31aad429c8460c1f0baf87073)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:50.601867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf2752-5d54-4df8-991b-6d7a568f76a3 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:50.603519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 582/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:05:50 2022] GET /identity/v3/users/860d90a31aad429c8460c1f0baf87073 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:50.609250 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033671f-0996-4cce-9f91-d9fbde82794b None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:50.651377 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:50.706947 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033671f-0996-4cce-9f91-d9fbde82794b None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:50.738569 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15e253-f027-4bc8-a36b-5c3d63e78a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:50.739229 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15e253-f027-4bc8-a36b-5c3d63e78a73 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:50.739520 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15e253-f027-4bc8-a36b-5c3d63e78a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:50.739798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15e253-f027-4bc8-a36b-5c3d63e78a73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:50.803604 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d15e253-f027-4bc8-a36b-5c3d63e78a73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:50.804982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d15e253-f027-4bc8-a36b-5c3d63e78a73 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:50.845152 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:50.898635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 601/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:05:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 09:05:50.905026 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf2752-5d54-4df8-991b-6d7a568f76a3 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:50.994789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:51.001042 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf2752-5d54-4df8-991b-6d7a568f76a3 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:51.033634 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2ae8a-2017-422d-b47a-a54195796c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:51.034489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2ae8a-2017-422d-b47a-a54195796c8b None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:51.035184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2ae8a-2017-422d-b47a-a54195796c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:51.035323 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2ae8a-2017-422d-b47a-a54195796c8b None admin] PATH_INFO: `/v3/projects/fec3b4beba25448ebfc0b77201706925` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:51.036501 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2ae8a-2017-422d-b47a-a54195796c8b None admin] RBAC: Authorizing `identity:get_project(project_id=fec3b4beba25448ebfc0b77201706925)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:51.037946 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2ae8a-2017-422d-b47a-a54195796c8b None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:51.040120 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 583/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:05:50 2022] GET /identity/v3/projects/fec3b4beba25448ebfc0b77201706925 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:51.046198 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15e253-f027-4bc8-a36b-5c3d63e78a73 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:51.140486 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15e253-f027-4bc8-a36b-5c3d63e78a73 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:51.172926 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a6075-d93b-4b8e-be75-91a1bd676382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:51.173581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a6075-d93b-4b8e-be75-91a1bd676382 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:51.173867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a6075-d93b-4b8e-be75-91a1bd676382 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:51.174145 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a6075-d93b-4b8e-be75-91a1bd676382 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:51.174995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a6075-d93b-4b8e-be75-91a1bd676382 None admin] RBAC: Adding query filter params (role.id=9cd080b50c6f4ee8a54bf9ef1655bd83, scope.project.id=fec3b4beba25448ebfc0b77201706925, user.id=860d90a31aad429c8460c1f0baf87073) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:51.175309 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a6075-d93b-4b8e-be75-91a1bd676382 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:51.176741 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a6075-d93b-4b8e-be75-91a1bd676382 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:51.182116 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 602/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:05:51 2022] GET /identity/v3/role_assignments?user.id=860d90a31aad429c8460c1f0baf87073&scope.project.id=fec3b4beba25448ebfc0b77201706925&role.id=9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:53.158013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f178b40-3bf8-4801-9289-a6d5445458cd None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:53.158452 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f178b40-3bf8-4801-9289-a6d5445458cd None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:53.158576 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f178b40-3bf8-4801-9289-a6d5445458cd None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:53.160004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 584/1186] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:53.165244 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a44198-42b5-47b6-8c75-c952a64dee84 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:53.165652 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a44198-42b5-47b6-8c75-c952a64dee84 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:53.165800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a44198-42b5-47b6-8c75-c952a64dee84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:53.176745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:53.182383 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-f0a44198-42b5-47b6-8c75-c952a64dee84 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:53.184088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-f0a44198-42b5-47b6-8c75-c952a64dee84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:53.248330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 603/1187] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:05:53.271663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f178b40-3bf8-4801-9289-a6d5445458cd None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:53.374718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f178b40-3bf8-4801-9289-a6d5445458cd None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:53.412037 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3476b-2688-4772-8b63-2e749f7c4cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:53.412749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3476b-2688-4772-8b63-2e749f7c4cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:53.413030 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3476b-2688-4772-8b63-2e749f7c4cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:53.413302 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3476b-2688-4772-8b63-2e749f7c4cc4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:53.423608 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3476b-2688-4772-8b63-2e749f7c4cc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:05:53.431277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-47b3476b-2688-4772-8b63-2e749f7c4cc4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:05:53.431277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:53.431277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:53.431277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:53.431277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:53.431277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:53.431277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:53.431277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:53.431277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:53.431277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:53.431277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:53.431277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:53.431277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:05:53.431277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:05:53.431277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:05:53.431277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:05:53.432435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:05:53.432435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:05:53.432435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:05:53.432435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:05:53.432435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:53.432435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:53.432435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:53.432435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:53.432435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:53.432435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:53.432435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:53.432435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:53.432435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:53.432435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:53.432435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:53.432435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:53.433541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:53.433541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:53.433541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:53.433541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:53.433541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:05:53.433541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:05:53.433541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:05:53.433541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:05:53.433541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:05:53.433541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:05:53.433541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:05:53.433541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:05:53.433541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 585/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:05:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:53.438632 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a44198-42b5-47b6-8c75-c952a64dee84 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:53.534432 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a44198-42b5-47b6-8c75-c952a64dee84 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:53.567251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b03809-773e-4cd6-968a-a869fcd28b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:53.567914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b03809-773e-4cd6-968a-a869fcd28b9a None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:53.568202 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b03809-773e-4cd6-968a-a869fcd28b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:53.568482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b03809-773e-4cd6-968a-a869fcd28b9a None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:53.569229 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b03809-773e-4cd6-968a-a869fcd28b9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:53.569536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b03809-773e-4cd6-968a-a869fcd28b9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:53.570897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b03809-773e-4cd6-968a-a869fcd28b9a None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:53.573047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:53.579674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 604/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:05:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:53.585379 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3476b-2688-4772-8b63-2e749f7c4cc4 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:53.611334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:53.686516 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3476b-2688-4772-8b63-2e749f7c4cc4 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:53.719356 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8a2be-4dcc-4fea-825f-7bbba1a9bd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:53.720212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8a2be-4dcc-4fea-825f-7bbba1a9bd5a None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:53.720500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8a2be-4dcc-4fea-825f-7bbba1a9bd5a None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:53.720780 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8a2be-4dcc-4fea-825f-7bbba1a9bd5a None admin] PATH_INFO: `/v3/users/860d90a31aad429c8460c1f0baf87073` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:53.721964 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8a2be-4dcc-4fea-825f-7bbba1a9bd5a None admin] RBAC: Authorizing `identity:get_user(user_id=860d90a31aad429c8460c1f0baf87073)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:53.723436 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8a2be-4dcc-4fea-825f-7bbba1a9bd5a None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:53.725064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 586/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:05:53 2022] GET /identity/v3/users/860d90a31aad429c8460c1f0baf87073 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:53.731365 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b03809-773e-4cd6-968a-a869fcd28b9a None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:53.825644 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b03809-773e-4cd6-968a-a869fcd28b9a None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:53.858101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ac468-de05-43f2-9792-2a665fb25b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:53.858960 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ac468-de05-43f2-9792-2a665fb25b2e None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:53.859248 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ac468-de05-43f2-9792-2a665fb25b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:53.859525 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ac468-de05-43f2-9792-2a665fb25b2e None admin] PATH_INFO: `/v3/projects/fec3b4beba25448ebfc0b77201706925` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:53.860677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ac468-de05-43f2-9792-2a665fb25b2e None admin] RBAC: Authorizing `identity:get_project(project_id=fec3b4beba25448ebfc0b77201706925)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:53.862098 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ac468-de05-43f2-9792-2a665fb25b2e None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:53.863792 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 605/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:05:53 2022] GET /identity/v3/projects/fec3b4beba25448ebfc0b77201706925 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:53.869609 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8a2be-4dcc-4fea-825f-7bbba1a9bd5a None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:53.957455 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:53.967424 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8a2be-4dcc-4fea-825f-7bbba1a9bd5a None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:54.000047 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10033f21-06db-4827-982f-34c0f07b3cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:54.000929 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10033f21-06db-4827-982f-34c0f07b3cba None admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:54.001217 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10033f21-06db-4827-982f-34c0f07b3cba None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:54.001492 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10033f21-06db-4827-982f-34c0f07b3cba None admin] PATH_INFO: `/v3/projects/fec3b4beba25448ebfc0b77201706925/users/860d90a31aad429c8460c1f0baf87073/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:54.003450 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10033f21-06db-4827-982f-34c0f07b3cba None admin] RBAC: Authorizing `identity:create_grant(project_id=fec3b4beba25448ebfc0b77201706925, user_id=860d90a31aad429c8460c1f0baf87073, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:54.004880 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10033f21-06db-4827-982f-34c0f07b3cba None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:54.084636 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 587/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:05:53 2022] PUT /identity/v3/projects/fec3b4beba25448ebfc0b77201706925/users/860d90a31aad429c8460c1f0baf87073/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:05:56.177546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c8a50-4501-4861-8101-fc3b24a1f0f2 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:56.177994 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c8a50-4501-4861-8101-fc3b24a1f0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:56.178110 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c8a50-4501-4861-8101-fc3b24a1f0f2 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:56.179501 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 606/1193] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:05:56.185173 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902c45e-2691-43a3-8d43-803fb53ffa55 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:56.185598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902c45e-2691-43a3-8d43-803fb53ffa55 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:56.185739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902c45e-2691-43a3-8d43-803fb53ffa55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:56.202761 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-1902c45e-2691-43a3-8d43-803fb53ffa55 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:05:56.203981 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-1902c45e-2691-43a3-8d43-803fb53ffa55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:05:56.268776 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 588/1194] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:05:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:05:56.293718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c8a50-4501-4861-8101-fc3b24a1f0f2 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:56.358049 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:56.394170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c8a50-4501-4861-8101-fc3b24a1f0f2 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:56.431294 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3df4b9-52b5-4093-849a-970d58dcc010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:56.432017 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3df4b9-52b5-4093-849a-970d58dcc010 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:56.432301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3df4b9-52b5-4093-849a-970d58dcc010 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:56.432612 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3df4b9-52b5-4093-849a-970d58dcc010 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:56.442870 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3df4b9-52b5-4093-849a-970d58dcc010 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:05:56.450482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-4e3df4b9-52b5-4093-849a-970d58dcc010 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:05:56.450482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:05:56.450482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:05:56.450482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:05:56.450482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:05:56.450482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:05:56.450482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:05:56.450482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:05:56.450482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:05:56.450482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:05:56.450482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:05:56.450482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:05:56.450482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:05:56.450482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:05:56.450482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:05:56.450482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:05:56.451681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:05:56.451681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:05:56.451681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:05:56.451681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:05:56.451681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:05:56.451681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:05:56.451681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:05:56.451681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:05:56.451681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:05:56.451681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:05:56.451681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:05:56.451681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:05:56.451681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:05:56.451681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:05:56.451681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:05:56.451681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:05:56.452956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:05:56.452956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:05:56.452956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:05:56.452956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:05:56.452956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:05:56.452956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:05:56.452956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:05:56.452956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:05:56.452956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:05:56.452956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:05:56.452956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:05:56.452956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:05:56.452956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 607/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:05:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:05:56.457724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902c45e-2691-43a3-8d43-803fb53ffa55 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:56.534183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:56.552653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902c45e-2691-43a3-8d43-803fb53ffa55 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:56.585373 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b8fbc-0661-466e-906b-4f878b776090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:56.586279 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b8fbc-0661-466e-906b-4f878b776090 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:56.586562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b8fbc-0661-466e-906b-4f878b776090 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:56.586837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b8fbc-0661-466e-906b-4f878b776090 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:56.587605 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b8fbc-0661-466e-906b-4f878b776090 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:56.587910 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b8fbc-0661-466e-906b-4f878b776090 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:56.589304 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b8fbc-0661-466e-906b-4f878b776090 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:56.596962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 589/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:05:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:56.602677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3df4b9-52b5-4093-849a-970d58dcc010 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:56.702108 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3df4b9-52b5-4093-849a-970d58dcc010 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:56.733971 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:56.742373 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63838d6-06b8-4c13-a4e1-ddf1a29a008c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:56.743283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63838d6-06b8-4c13-a4e1-ddf1a29a008c None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:56.743575 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63838d6-06b8-4c13-a4e1-ddf1a29a008c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:56.747830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63838d6-06b8-4c13-a4e1-ddf1a29a008c None admin] PATH_INFO: `/v3/users/860d90a31aad429c8460c1f0baf87073` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:56.747830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63838d6-06b8-4c13-a4e1-ddf1a29a008c None admin] RBAC: Authorizing `identity:get_user(user_id=860d90a31aad429c8460c1f0baf87073)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:56.747830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63838d6-06b8-4c13-a4e1-ddf1a29a008c None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:56.748318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 608/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jun 23 09:05:56 2022] GET /identity/v3/users/860d90a31aad429c8460c1f0baf87073 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:56.753965 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b8fbc-0661-466e-906b-4f878b776090 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:56.851720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b8fbc-0661-466e-906b-4f878b776090 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:56.884137 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bb6bf-558f-4c65-ab05-5f0476a8810f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:56.884783 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bb6bf-558f-4c65-ab05-5f0476a8810f None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:56.885073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bb6bf-558f-4c65-ab05-5f0476a8810f None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:56.885357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bb6bf-558f-4c65-ab05-5f0476a8810f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:56.896746 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:56.950806 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659bb6bf-558f-4c65-ab05-5f0476a8810f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:56.952343 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659bb6bf-558f-4c65-ab05-5f0476a8810f None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:57.049116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 590/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:05:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 09:05:57.055028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63838d6-06b8-4c13-a4e1-ddf1a29a008c None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:57.160595 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63838d6-06b8-4c13-a4e1-ddf1a29a008c None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:57.193195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c6997-cb89-4566-adef-c4888c2417cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:57.194394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c6997-cb89-4566-adef-c4888c2417cd None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:57.194683 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c6997-cb89-4566-adef-c4888c2417cd None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:57.194991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c6997-cb89-4566-adef-c4888c2417cd None admin] PATH_INFO: `/v3/projects/fec3b4beba25448ebfc0b77201706925` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:57.196172 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c6997-cb89-4566-adef-c4888c2417cd None admin] RBAC: Authorizing `identity:get_project(project_id=fec3b4beba25448ebfc0b77201706925)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:57.197611 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c6997-cb89-4566-adef-c4888c2417cd None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:57.199855 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 609/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jun 23 09:05:57 2022] GET /identity/v3/projects/fec3b4beba25448ebfc0b77201706925 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:05:57.206121 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bb6bf-558f-4c65-ab05-5f0476a8810f None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:05:57.249595 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:05:57.305270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bb6bf-558f-4c65-ab05-5f0476a8810f None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:05:57.340119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc99bb-6eb7-4360-a1f2-6f8e68b98627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:05:57.340764 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc99bb-6eb7-4360-a1f2-6f8e68b98627 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:05:57.341045 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc99bb-6eb7-4360-a1f2-6f8e68b98627 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:05:57.341317 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc99bb-6eb7-4360-a1f2-6f8e68b98627 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:05:57.342141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc99bb-6eb7-4360-a1f2-6f8e68b98627 None admin] RBAC: Adding query filter params (role.id=9cd080b50c6f4ee8a54bf9ef1655bd83, scope.project.id=fec3b4beba25448ebfc0b77201706925, user.id=860d90a31aad429c8460c1f0baf87073) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:05:57.342452 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc99bb-6eb7-4360-a1f2-6f8e68b98627 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:05:57.343910 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc99bb-6eb7-4360-a1f2-6f8e68b98627 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:05:57.349623 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 591/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:05:57 2022] GET /identity/v3/role_assignments?user.id=860d90a31aad429c8460c1f0baf87073&scope.project.id=fec3b4beba25448ebfc0b77201706925&role.id=9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:06:07.189023 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=69 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:06:07.347049 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=47 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:16.217640 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74dbdaf-139b-4027-8ee3-fd9be20c0b6f None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:16.218216 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74dbdaf-139b-4027-8ee3-fd9be20c0b6f None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:16.218216 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74dbdaf-139b-4027-8ee3-fd9be20c0b6f None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:16.219615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 610/1201] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:07:16.224874 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d2aa6-a904-49d7-8204-e572b778ad83 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:16.225328 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d2aa6-a904-49d7-8204-e572b778ad83 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:16.225502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d2aa6-a904-49d7-8204-e572b778ad83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:16.242303 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-466d2aa6-a904-49d7-8204-e572b778ad83 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:07:16.243554 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-466d2aa6-a904-49d7-8204-e572b778ad83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:07:16.309817 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 592/1202] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:07:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:07:16.334786 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74dbdaf-139b-4027-8ee3-fd9be20c0b6f None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:16.442401 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74dbdaf-139b-4027-8ee3-fd9be20c0b6f None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:16.476763 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60a95b-0872-4719-b57b-87e006d4ddc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:16.477588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60a95b-0872-4719-b57b-87e006d4ddc7 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:16.477873 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60a95b-0872-4719-b57b-87e006d4ddc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:16.478152 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60a95b-0872-4719-b57b-87e006d4ddc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:16.488705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60a95b-0872-4719-b57b-87e006d4ddc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:16.490136 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60a95b-0872-4719-b57b-87e006d4ddc7 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:16.508436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-7e60a95b-0872-4719-b57b-87e006d4ddc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:16.508436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:16.508436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:16.508436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:16.508436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:16.508436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:16.508436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:16.508436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:16.508436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:16.508436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:16.508436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:16.508436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:16.508436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:07:16.508436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:07:16.508436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:07:16.509702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:07:16.509702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:16.509702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:16.509702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:16.509702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:16.509702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:16.509702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:16.509702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:16.509702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:16.509702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:16.509702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:16.509702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:16.509702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:16.509702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:16.509702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:16.509702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:16.509702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:16.511407 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:07:16.511407 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:07:16.511407 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:16.511407 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:07:16.511407 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 611/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:07:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:16.516897 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d2aa6-a904-49d7-8204-e572b778ad83 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:16.622427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d2aa6-a904-49d7-8204-e572b778ad83 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:16.651292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:16.656931 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0886047-bbe8-435e-a646-dbb0565cc067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:16.657596 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0886047-bbe8-435e-a646-dbb0565cc067 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:16.657888 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0886047-bbe8-435e-a646-dbb0565cc067 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:16.658168 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0886047-bbe8-435e-a646-dbb0565cc067 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:16.658892 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0886047-bbe8-435e-a646-dbb0565cc067 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:16.659234 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0886047-bbe8-435e-a646-dbb0565cc067 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:16.660674 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0886047-bbe8-435e-a646-dbb0565cc067 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:16.671699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 593/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:16.677429 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60a95b-0872-4719-b57b-87e006d4ddc7 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:16.731254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:16.777162 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60a95b-0872-4719-b57b-87e006d4ddc7 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:16.810032 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257356c6-2a0c-4e2d-8e6d-fcb0b561d2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:16.810710 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257356c6-2a0c-4e2d-8e6d-fcb0b561d2e3 None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:16.811038 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257356c6-2a0c-4e2d-8e6d-fcb0b561d2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:16.811317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257356c6-2a0c-4e2d-8e6d-fcb0b561d2e3 None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:16.812035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257356c6-2a0c-4e2d-8e6d-fcb0b561d2e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:16.813473 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257356c6-2a0c-4e2d-8e6d-fcb0b561d2e3 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:17.016264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 612/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jun 23 09:07:16 2022] POST /identity/v3/users => generated 251 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:07:19.106975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b5e3c-212e-4d0c-ae12-23757ed902dd None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:19.116087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b5e3c-212e-4d0c-ae12-23757ed902dd None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:19.116087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b5e3c-212e-4d0c-ae12-23757ed902dd None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:19.116087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 594/1206] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:07:19.116380 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c64f3-b967-4c33-b5ce-1c5c5dfaf0a0 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:19.116380 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c64f3-b967-4c33-b5ce-1c5c5dfaf0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:19.116380 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c64f3-b967-4c33-b5ce-1c5c5dfaf0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:19.132254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-6a4c64f3-b967-4c33-b5ce-1c5c5dfaf0a0 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:07:19.133428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-6a4c64f3-b967-4c33-b5ce-1c5c5dfaf0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:07:19.203675 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 613/1207] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:07:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:07:19.227497 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b5e3c-212e-4d0c-ae12-23757ed902dd None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:19.334245 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b5e3c-212e-4d0c-ae12-23757ed902dd None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:19.367124 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2671216-d7c9-418f-bfd5-448b5cc6bc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:19.367849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2671216-d7c9-418f-bfd5-448b5cc6bc87 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:19.368135 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2671216-d7c9-418f-bfd5-448b5cc6bc87 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:19.368414 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2671216-d7c9-418f-bfd5-448b5cc6bc87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:19.379264 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2671216-d7c9-418f-bfd5-448b5cc6bc87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:07:19.387489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-e2671216-d7c9-418f-bfd5-448b5cc6bc87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:07:19.387489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:19.387489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:19.387489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:19.387489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:19.387489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:19.387489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:19.387489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:19.387489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:19.387489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:19.387489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:19.387489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:19.387489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:07:19.387489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:07:19.387489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:07:19.387489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:07:19.397306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:07:19.397306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:07:19.397306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:07:19.397306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:07:19.397306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:19.397306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:19.397306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:19.397306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:19.397306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:19.397306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:19.397306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:19.397306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:19.397306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:19.397306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:19.397306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:19.397306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:19.398227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:19.398227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:19.398227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:19.398227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:19.398227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:07:19.398227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:07:19.398227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:07:19.398227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:07:19.398227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:07:19.398227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:07:19.398227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:07:19.398227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:19.398227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 595/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:07:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:19.399061 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c64f3-b967-4c33-b5ce-1c5c5dfaf0a0 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:19.437053 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:19.494430 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c64f3-b967-4c33-b5ce-1c5c5dfaf0a0 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:19.532405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6d4c7-6577-4d21-ad9b-0363a096aaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:19.533086 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6d4c7-6577-4d21-ad9b-0363a096aaf6 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:19.536499 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6d4c7-6577-4d21-ad9b-0363a096aaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:19.536499 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6d4c7-6577-4d21-ad9b-0363a096aaf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:19.536499 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6d4c7-6577-4d21-ad9b-0363a096aaf6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:19.536499 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6d4c7-6577-4d21-ad9b-0363a096aaf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:19.536890 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6d4c7-6577-4d21-ad9b-0363a096aaf6 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:19.545248 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 614/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jun 23 09:07:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:07:19.550941 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2671216-d7c9-418f-bfd5-448b5cc6bc87 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:19.601356 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:19.650733 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2671216-d7c9-418f-bfd5-448b5cc6bc87 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:19.684917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36dc20f-4392-46f8-8f21-26ff9888acb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:19.685700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36dc20f-4392-46f8-8f21-26ff9888acb3 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:19.685985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36dc20f-4392-46f8-8f21-26ff9888acb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:19.686262 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36dc20f-4392-46f8-8f21-26ff9888acb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:19.695940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36dc20f-4392-46f8-8f21-26ff9888acb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:19.697392 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36dc20f-4392-46f8-8f21-26ff9888acb3 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:19.715005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-f36dc20f-4392-46f8-8f21-26ff9888acb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:19.715005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:19.715005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:19.715005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:19.715005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:19.715005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:19.715005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:19.715005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:19.715005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:19.715005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:19.715005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:19.715005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:19.715005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:07:19.715005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:07:19.715005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:07:19.716141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:07:19.716141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:19.716141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:19.716141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:19.716141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:19.716141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:19.716141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:19.716141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:19.716141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:19.716141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:19.716141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:19.716141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:19.716141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:19.716141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:19.716141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:19.716141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:19.716141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:19.717229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:07:19.717229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:07:19.717229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:19.717229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:19.717229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 596/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:07:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:19.721912 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6d4c7-6577-4d21-ad9b-0363a096aaf6 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:19.820464 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:19.831348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6d4c7-6577-4d21-ad9b-0363a096aaf6 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:19.867056 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007a4c4-3377-4f77-8b77-6867dbb6130a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:19.867727 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007a4c4-3377-4f77-8b77-6867dbb6130a None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:19.868013 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007a4c4-3377-4f77-8b77-6867dbb6130a None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:19.868289 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007a4c4-3377-4f77-8b77-6867dbb6130a None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:19.869009 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9007a4c4-3377-4f77-8b77-6867dbb6130a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:19.869800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9007a4c4-3377-4f77-8b77-6867dbb6130a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:19.871274 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9007a4c4-3377-4f77-8b77-6867dbb6130a None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:19.888266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 615/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:19.893951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36dc20f-4392-46f8-8f21-26ff9888acb3 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:19.995380 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36dc20f-4392-46f8-8f21-26ff9888acb3 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:19.997266 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:20.029187 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f8ba1-a266-4fda-bda2-6601c46b3cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:20.029971 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f8ba1-a266-4fda-bda2-6601c46b3cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:20.034367 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f8ba1-a266-4fda-bda2-6601c46b3cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:20.034367 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f8ba1-a266-4fda-bda2-6601c46b3cc3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:20.037440 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f8ba1-a266-4fda-bda2-6601c46b3cc3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:20.039386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f8ba1-a266-4fda-bda2-6601c46b3cc3 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:20.053689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-6b1f8ba1-a266-4fda-bda2-6601c46b3cc3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 09:07:20.053689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:20.053689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:20.053689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:20.053689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:20.053689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:20.053689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:20.053689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:20.053689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:20.053689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:20.053689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:20.053689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:20.053689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:07:20.053689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:07:20.053689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:07:20.053689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:07:20.054867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:20.054867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:20.054867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:07:20.054867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:07:20.054867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:07:20.054867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:07:20.054867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:20.054867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:20.054867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:20.054867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:20.054867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:20.054867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:20.054867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:20.054867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:20.054867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:20.054867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:20.054867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:20.054867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:20.055980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:20.055980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:20.055980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:07:20.055980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:07:20.055980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:07:20.055980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:07:20.055980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:07:20.055980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:07:20.055980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 09:07:20.055980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:20.055980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 597/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:07:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:20.061153 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007a4c4-3377-4f77-8b77-6867dbb6130a None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:20.161975 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007a4c4-3377-4f77-8b77-6867dbb6130a None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:20.196887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b14fc0-e245-48f7-91b9-065b16be339a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:20.197599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b14fc0-e245-48f7-91b9-065b16be339a None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:20.197892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b14fc0-e245-48f7-91b9-065b16be339a None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:20.198227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b14fc0-e245-48f7-91b9-065b16be339a None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:20.199056 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b14fc0-e245-48f7-91b9-065b16be339a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:20.199375 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b14fc0-e245-48f7-91b9-065b16be339a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:20.200817 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b14fc0-e245-48f7-91b9-065b16be339a None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:20.220231 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 616/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jun 23 09:07:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:20.226069 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f8ba1-a266-4fda-bda2-6601c46b3cc3 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:20.324889 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f8ba1-a266-4fda-bda2-6601c46b3cc3 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:20.359045 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71407b5b-af0d-4bd7-96fd-2dfb8795952f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:20.359621 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71407b5b-af0d-4bd7-96fd-2dfb8795952f None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:20.359906 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71407b5b-af0d-4bd7-96fd-2dfb8795952f None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:20.360185 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71407b5b-af0d-4bd7-96fd-2dfb8795952f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:20.389420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:20.426213 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71407b5b-af0d-4bd7-96fd-2dfb8795952f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:20.427708 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71407b5b-af0d-4bd7-96fd-2dfb8795952f None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:20.528986 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 598/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:07:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 09:07:20.534871 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b14fc0-e245-48f7-91b9-065b16be339a None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:20.553924 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:20.634526 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b14fc0-e245-48f7-91b9-065b16be339a None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:20.668614 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a1d3a-79ab-4ff9-8bc6-7cbec6b38264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:20.669404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a1d3a-79ab-4ff9-8bc6-7cbec6b38264 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:20.669693 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a1d3a-79ab-4ff9-8bc6-7cbec6b38264 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:20.669969 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a1d3a-79ab-4ff9-8bc6-7cbec6b38264 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:20.680458 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a1d3a-79ab-4ff9-8bc6-7cbec6b38264 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:20.682361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a1d3a-79ab-4ff9-8bc6-7cbec6b38264 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:20.700348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-f33a1d3a-79ab-4ff9-8bc6-7cbec6b38264 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:20.700348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:20.700348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:20.700348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:20.700348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:20.700348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:20.700348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:20.700348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:20.700348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:20.700348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:20.700348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:20.700348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:20.700348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:07:20.700348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:07:20.700348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:07:20.701588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:07:20.701588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:20.701588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:20.701588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:20.701588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:20.701588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:20.701588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:20.701588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:20.701588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:20.701588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:20.701588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:20.701588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:20.701588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:20.701588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:20.701588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:20.701588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:20.701588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:20.702882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:07:20.702882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:07:20.702882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:20.702882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:07:20.702882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 617/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:07:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:20.707492 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71407b5b-af0d-4bd7-96fd-2dfb8795952f None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:20.768105 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:20.804289 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71407b5b-af0d-4bd7-96fd-2dfb8795952f None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:20.837536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d68dfa-2f66-4b6d-bd1c-e26a1165f629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:20.838477 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d68dfa-2f66-4b6d-bd1c-e26a1165f629 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:20.838762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d68dfa-2f66-4b6d-bd1c-e26a1165f629 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:20.839071 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d68dfa-2f66-4b6d-bd1c-e26a1165f629 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:20.839788 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d68dfa-2f66-4b6d-bd1c-e26a1165f629 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:20.840090 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d68dfa-2f66-4b6d-bd1c-e26a1165f629 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:20.841457 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d68dfa-2f66-4b6d-bd1c-e26a1165f629 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:20.853265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 599/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:20.858935 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a1d3a-79ab-4ff9-8bc6-7cbec6b38264 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:20.929090 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:20.958437 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a1d3a-79ab-4ff9-8bc6-7cbec6b38264 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:20.994114 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ee99d-8c7a-4db2-bfa0-5651e6d33ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:21.006306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ee99d-8c7a-4db2-bfa0-5651e6d33ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:21.006306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ee99d-8c7a-4db2-bfa0-5651e6d33ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:21.006306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ee99d-8c7a-4db2-bfa0-5651e6d33ff6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:21.006306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ee99d-8c7a-4db2-bfa0-5651e6d33ff6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:21.006306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ee99d-8c7a-4db2-bfa0-5651e6d33ff6 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:21.018399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-e67ee99d-8c7a-4db2-bfa0-5651e6d33ff6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:07:21.018399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:21.018399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:21.018399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:21.018399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:21.018399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:21.018399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:21.018399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:21.018399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:21.018399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:21.018399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:21.018399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:21.018399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:07:21.018399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:07:21.018399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:07:21.019539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:07:21.019539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:21.019539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:21.019539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:21.019539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:21.019539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:21.019539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:21.019539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:21.019539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:21.019539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:21.019539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:21.019539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:21.019539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:21.019539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:21.019539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:21.019539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:21.019539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:21.020999 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:07:21.020999 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:07:21.020999 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:07:21.020999 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:07:21.020999 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:07:21.020999 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:07:21.020999 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:07:21.020999 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:07:21.020999 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 618/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:07:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:21.025623 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d68dfa-2f66-4b6d-bd1c-e26a1165f629 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:21.123408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d68dfa-2f66-4b6d-bd1c-e26a1165f629 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:21.147023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:21.158774 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dff60a-9ae6-4e53-b07d-3a7f9a79b42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:21.159469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dff60a-9ae6-4e53-b07d-3a7f9a79b42d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:21.159758 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dff60a-9ae6-4e53-b07d-3a7f9a79b42d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:21.165317 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dff60a-9ae6-4e53-b07d-3a7f9a79b42d None admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:21.165317 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dff60a-9ae6-4e53-b07d-3a7f9a79b42d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:21.165317 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dff60a-9ae6-4e53-b07d-3a7f9a79b42d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:21.165317 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dff60a-9ae6-4e53-b07d-3a7f9a79b42d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:21.174482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 600/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:07:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:21.180720 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ee99d-8c7a-4db2-bfa0-5651e6d33ff6 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:21.287081 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ee99d-8c7a-4db2-bfa0-5651e6d33ff6 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:21.314085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:21.319841 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505b1b5-fcc9-4df6-afb5-0bc2a04b185a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:21.320501 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505b1b5-fcc9-4df6-afb5-0bc2a04b185a None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:21.320793 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505b1b5-fcc9-4df6-afb5-0bc2a04b185a None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:21.321074 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505b1b5-fcc9-4df6-afb5-0bc2a04b185a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:21.321915 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505b1b5-fcc9-4df6-afb5-0bc2a04b185a None admin] RBAC: Adding query filter params (role.id=edf696ae763147cab68434524d1eff9c, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=98a63b20ceca40e6bceb56eda8ae9686) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:21.322234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505b1b5-fcc9-4df6-afb5-0bc2a04b185a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:21.324216 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505b1b5-fcc9-4df6-afb5-0bc2a04b185a None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:21.329760 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 619/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:07:21 2022] GET /identity/v3/role_assignments?user.id=98a63b20ceca40e6bceb56eda8ae9686&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=edf696ae763147cab68434524d1eff9c => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:23.427081 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24777c2e-b685-4dec-a6f5-ddce44a7e023 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:23.427607 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24777c2e-b685-4dec-a6f5-ddce44a7e023 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:23.427607 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24777c2e-b685-4dec-a6f5-ddce44a7e023 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:23.428929 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 601/1220] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:07:23.434373 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1b7cb-ac0b-46a5-b9ce-8a1458ed8a86 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:23.434774 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1b7cb-ac0b-46a5-b9ce-8a1458ed8a86 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:23.434998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1b7cb-ac0b-46a5-b9ce-8a1458ed8a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:23.452202 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-ccd1b7cb-ac0b-46a5-b9ce-8a1458ed8a86 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:07:23.453393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-ccd1b7cb-ac0b-46a5-b9ce-8a1458ed8a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:07:23.520506 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 620/1221] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:07:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:07:23.544160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24777c2e-b685-4dec-a6f5-ddce44a7e023 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:23.643999 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24777c2e-b685-4dec-a6f5-ddce44a7e023 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:23.677331 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487eb70-172d-4fe3-9e93-4f7a955b7bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:23.678051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487eb70-172d-4fe3-9e93-4f7a955b7bde None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:23.678334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487eb70-172d-4fe3-9e93-4f7a955b7bde None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:23.678611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487eb70-172d-4fe3-9e93-4f7a955b7bde None admin] PATH_INFO: `/v3/roles/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:23.688606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487eb70-172d-4fe3-9e93-4f7a955b7bde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:07:23.696292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-1487eb70-172d-4fe3-9e93-4f7a955b7bde None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:07:23.696292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:23.696292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:23.696292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:23.696292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:23.696292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:23.696292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:23.696292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:23.696292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:23.696292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:23.696292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:23.696292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:23.696292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:07:23.696292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:07:23.696292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:07:23.696292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:07:23.697332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:07:23.697332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:07:23.697332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:07:23.697332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:07:23.697332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:23.697332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:23.697332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:23.697332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:23.697332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:23.697332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:23.697332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:23.697332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:23.697332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:23.697332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:23.697332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:23.697332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:23.698341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:23.698341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:23.698341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:23.698341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:23.698341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:07:23.698341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:07:23.698341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:07:23.698341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:07:23.698341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:07:23.698341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:07:23.698341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:07:23.698341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:23.698341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 602/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:07:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:23.703941 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1b7cb-ac0b-46a5-b9ce-8a1458ed8a86 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:23.801375 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1b7cb-ac0b-46a5-b9ce-8a1458ed8a86 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:23.824353 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:23.834592 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791555c7-7e1a-411b-8630-53f52e852c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:23.835302 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791555c7-7e1a-411b-8630-53f52e852c68 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:23.835594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791555c7-7e1a-411b-8630-53f52e852c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:23.835875 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791555c7-7e1a-411b-8630-53f52e852c68 None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:23.836624 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791555c7-7e1a-411b-8630-53f52e852c68 None admin] RBAC: Adding query filter params (name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:23.836938 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791555c7-7e1a-411b-8630-53f52e852c68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:23.838903 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791555c7-7e1a-411b-8630-53f52e852c68 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:23.846796 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 621/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jun 23 09:07:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:23.852536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487eb70-172d-4fe3-9e93-4f7a955b7bde None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:23.895511 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:23.953602 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487eb70-172d-4fe3-9e93-4f7a955b7bde None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:23.990033 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad68fa6-4932-44d3-b512-4455f5a55f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:23.990829 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad68fa6-4932-44d3-b512-4455f5a55f66 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:23.991142 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad68fa6-4932-44d3-b512-4455f5a55f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:23.991423 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad68fa6-4932-44d3-b512-4455f5a55f66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:24.003433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad68fa6-4932-44d3-b512-4455f5a55f66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:24.004912 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad68fa6-4932-44d3-b512-4455f5a55f66 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:24.021881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-aad68fa6-4932-44d3-b512-4455f5a55f66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:24.021881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:24.021881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:24.021881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:24.021881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:24.021881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:24.021881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:24.021881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:24.021881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:24.021881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:24.021881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:24.021881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:24.021881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:07:24.021881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:07:24.021881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:07:24.026973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:07:24.026973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:24.026973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:24.026973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:24.026973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:24.026973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:24.026973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:24.026973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:24.026973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:24.026973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:24.026973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:24.026973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:24.026973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:24.026973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:24.026973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:24.026973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:24.026973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:24.028165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:07:24.028165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:07:24.028165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:24.028165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:24.028165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 603/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:24.029211 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791555c7-7e1a-411b-8630-53f52e852c68 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:24.128359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791555c7-7e1a-411b-8630-53f52e852c68 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:24.161084 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934acbc9-6a72-4e29-a82e-74bba00f98bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:24.161752 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934acbc9-6a72-4e29-a82e-74bba00f98bb None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:24.162036 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934acbc9-6a72-4e29-a82e-74bba00f98bb None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:24.162315 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934acbc9-6a72-4e29-a82e-74bba00f98bb None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:24.163080 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934acbc9-6a72-4e29-a82e-74bba00f98bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:24.163386 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934acbc9-6a72-4e29-a82e-74bba00f98bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:24.164871 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934acbc9-6a72-4e29-a82e-74bba00f98bb None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:24.177525 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 622/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:24.183925 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad68fa6-4932-44d3-b512-4455f5a55f66 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:24.280044 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:24.289692 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad68fa6-4932-44d3-b512-4455f5a55f66 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:24.329175 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aeb720-e951-4ef6-8112-b87f74065165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:24.329980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aeb720-e951-4ef6-8112-b87f74065165 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:24.330268 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aeb720-e951-4ef6-8112-b87f74065165 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:24.330543 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aeb720-e951-4ef6-8112-b87f74065165 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:24.338423 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aeb720-e951-4ef6-8112-b87f74065165 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:24.340408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aeb720-e951-4ef6-8112-b87f74065165 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:24.354639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-51aeb720-e951-4ef6-8112-b87f74065165 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 09:07:24.354639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:24.354639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:24.354639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:24.354639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:24.354639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:24.354639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:24.354639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:24.354639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:24.354639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:24.354639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:24.354639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:24.354639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:07:24.354639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:07:24.354639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:07:24.354639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:07:24.360863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:24.360863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:24.360863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:07:24.360863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:07:24.360863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:07:24.360863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:07:24.360863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:24.360863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:24.360863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:24.360863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:24.360863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:24.360863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:24.360863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:24.360863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:24.360863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:24.360863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:24.360863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:24.360863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:24.361922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:24.361922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:24.361922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:07:24.361922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:07:24.361922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:07:24.361922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:07:24.361922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:07:24.361922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:07:24.361922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 09:07:24.361922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:24.361922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 604/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:07:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:24.365443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934acbc9-6a72-4e29-a82e-74bba00f98bb None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:24.412963 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:24.464198 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934acbc9-6a72-4e29-a82e-74bba00f98bb None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:24.497738 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34cc70-db1f-40bd-ba89-07525cd9ca54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:24.498414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34cc70-db1f-40bd-ba89-07525cd9ca54 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:24.498703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34cc70-db1f-40bd-ba89-07525cd9ca54 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:24.499013 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34cc70-db1f-40bd-ba89-07525cd9ca54 None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:24.499860 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34cc70-db1f-40bd-ba89-07525cd9ca54 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:24.500260 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34cc70-db1f-40bd-ba89-07525cd9ca54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:24.501699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34cc70-db1f-40bd-ba89-07525cd9ca54 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:24.520311 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 623/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jun 23 09:07:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:24.525939 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aeb720-e951-4ef6-8112-b87f74065165 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:24.625453 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aeb720-e951-4ef6-8112-b87f74065165 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:24.660781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1376df-10c9-4396-9349-78b28195fdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:24.661568 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1376df-10c9-4396-9349-78b28195fdda None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:24.661856 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1376df-10c9-4396-9349-78b28195fdda None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:24.662135 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1376df-10c9-4396-9349-78b28195fdda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:24.672989 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1376df-10c9-4396-9349-78b28195fdda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:24.674439 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1376df-10c9-4396-9349-78b28195fdda None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:24.690893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-9b1376df-10c9-4396-9349-78b28195fdda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:24.690893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:24.690893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:24.690893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:24.690893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:24.690893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:24.690893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:24.690893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:24.690893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:24.690893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:24.690893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:24.690893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:24.690893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:07:24.690893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:07:24.690893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:07:24.692015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:07:24.692015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:24.692015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:24.692015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:24.692015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:24.692015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:24.692015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:24.692015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:24.692015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:24.692015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:24.692015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:24.692015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:24.692015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:24.692015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:24.692015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:24.692015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:24.692015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:24.693495 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:07:24.693495 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:07:24.693495 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:24.693495 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:24.693495 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 605/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:07:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:24.697804 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34cc70-db1f-40bd-ba89-07525cd9ca54 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:24.797614 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34cc70-db1f-40bd-ba89-07525cd9ca54 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:24.799944 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:24.831524 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886e415-d6c1-41cf-8e0d-3c279d8a2910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:24.832187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886e415-d6c1-41cf-8e0d-3c279d8a2910 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:24.832475 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886e415-d6c1-41cf-8e0d-3c279d8a2910 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:24.832758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886e415-d6c1-41cf-8e0d-3c279d8a2910 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:24.833480 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b886e415-d6c1-41cf-8e0d-3c279d8a2910 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:24.833786 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b886e415-d6c1-41cf-8e0d-3c279d8a2910 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:24.835182 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b886e415-d6c1-41cf-8e0d-3c279d8a2910 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:24.846990 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 624/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:24.852792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1376df-10c9-4396-9349-78b28195fdda None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:24.862591 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:24.952433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1376df-10c9-4396-9349-78b28195fdda None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:24.985733 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccfa30-5df5-4df4-9260-08b1f87edc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:24.986494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccfa30-5df5-4df4-9260-08b1f87edc47 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:24.986782 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccfa30-5df5-4df4-9260-08b1f87edc47 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:24.987092 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccfa30-5df5-4df4-9260-08b1f87edc47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:24.993630 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccfa30-5df5-4df4-9260-08b1f87edc47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:24.995110 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccfa30-5df5-4df4-9260-08b1f87edc47 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:25.009399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-57ccfa30-5df5-4df4-9260-08b1f87edc47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:07:25.009399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:25.009399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:25.009399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:25.009399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:25.009399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:25.009399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:25.009399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:25.009399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:25.009399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:25.009399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:25.009399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:25.009399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:07:25.009399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:07:25.009399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:07:25.011051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:07:25.011051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:25.011051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:25.011051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:25.011051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:25.011051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:25.011051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:25.011051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:25.011051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:25.011051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:25.011051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:25.011051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:25.011051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:25.011051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:25.011051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:25.011051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:25.011051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:25.012406 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:07:25.012406 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:07:25.012406 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:07:25.012406 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:07:25.012406 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:07:25.012406 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:07:25.012406 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:07:25.012406 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:25.012406 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 606/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:07:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:25.016897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886e415-d6c1-41cf-8e0d-3c279d8a2910 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:25.115669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886e415-d6c1-41cf-8e0d-3c279d8a2910 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:25.148196 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f75c52-b5c2-4463-bded-c4556ce9d366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:25.149010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f75c52-b5c2-4463-bded-c4556ce9d366 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:25.149221 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f75c52-b5c2-4463-bded-c4556ce9d366 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:25.149465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f75c52-b5c2-4463-bded-c4556ce9d366 None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:25.150264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f75c52-b5c2-4463-bded-c4556ce9d366 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:25.151132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f75c52-b5c2-4463-bded-c4556ce9d366 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:25.152583 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f75c52-b5c2-4463-bded-c4556ce9d366 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:25.164667 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 625/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:07:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:25.171198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccfa30-5df5-4df4-9260-08b1f87edc47 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:25.235556 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:25.268169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccfa30-5df5-4df4-9260-08b1f87edc47 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:25.302081 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf5a20-eb9f-4bc4-8a07-2413c02fbc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:25.302986 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf5a20-eb9f-4bc4-8a07-2413c02fbc78 None admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:25.303276 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf5a20-eb9f-4bc4-8a07-2413c02fbc78 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:25.303553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf5a20-eb9f-4bc4-8a07-2413c02fbc78 None admin] PATH_INFO: `/v3/projects/93b5040533524222b126bbe93afc56d1/users/98a63b20ceca40e6bceb56eda8ae9686/roles/edf696ae763147cab68434524d1eff9c` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:25.317147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf5a20-eb9f-4bc4-8a07-2413c02fbc78 None admin] RBAC: Authorizing `identity:create_grant(project_id=93b5040533524222b126bbe93afc56d1, user_id=98a63b20ceca40e6bceb56eda8ae9686, role_id=edf696ae763147cab68434524d1eff9c)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:25.321800 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf5a20-eb9f-4bc4-8a07-2413c02fbc78 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:25.472711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 607/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:07:25 2022] PUT /identity/v3/projects/93b5040533524222b126bbe93afc56d1/users/98a63b20ceca40e6bceb56eda8ae9686/roles/edf696ae763147cab68434524d1eff9c => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:07:27.505841 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9c15a-2f62-4d82-ab1f-81f7165f28cc None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:27.506402 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9c15a-2f62-4d82-ab1f-81f7165f28cc None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:27.506402 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9c15a-2f62-4d82-ab1f-81f7165f28cc None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:27.507744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 626/1233] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:07:27.513495 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25d79e-628d-4703-a304-98b472dd994b None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:27.513914 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25d79e-628d-4703-a304-98b472dd994b None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:27.514057 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25d79e-628d-4703-a304-98b472dd994b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:27.530701 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-5d25d79e-628d-4703-a304-98b472dd994b None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:07:27.531917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5d25d79e-628d-4703-a304-98b472dd994b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:07:27.598433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 608/1234] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:07:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:07:27.623652 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9c15a-2f62-4d82-ab1f-81f7165f28cc None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:27.648792 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:27.723028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9c15a-2f62-4d82-ab1f-81f7165f28cc None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:27.756941 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537acee7-0e7c-4af6-90a3-92cae6f51eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:27.757791 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537acee7-0e7c-4af6-90a3-92cae6f51eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:27.758082 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537acee7-0e7c-4af6-90a3-92cae6f51eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:27.758363 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537acee7-0e7c-4af6-90a3-92cae6f51eb2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:27.769249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537acee7-0e7c-4af6-90a3-92cae6f51eb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:07:27.777427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-537acee7-0e7c-4af6-90a3-92cae6f51eb2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:07:27.777427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:27.777427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:27.777427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:27.777427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:27.777427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:27.777427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:27.777427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:27.777427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:27.777427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:27.777427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:27.777427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:27.777427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:07:27.777427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:07:27.777427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:07:27.777427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:07:27.778601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:07:27.778601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:07:27.778601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:07:27.778601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:07:27.778601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:27.778601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:27.778601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:27.778601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:27.778601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:27.778601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:27.778601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:27.778601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:27.778601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:27.778601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:27.778601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:27.778601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:27.780237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:27.780237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:27.780237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:27.780237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:27.780237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:07:27.780237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:07:27.780237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:07:27.780237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:07:27.780237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:07:27.780237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:07:27.780237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 09:07:27.780237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:07:27.780237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 627/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jun 23 09:07:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:27.784607 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25d79e-628d-4703-a304-98b472dd994b None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:27.832045 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:27.882897 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25d79e-628d-4703-a304-98b472dd994b None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:27.917881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2a5a1-78b5-45d2-ad9a-a1a8d0964970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:27.918548 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2a5a1-78b5-45d2-ad9a-a1a8d0964970 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:27.918835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2a5a1-78b5-45d2-ad9a-a1a8d0964970 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:27.919148 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2a5a1-78b5-45d2-ad9a-a1a8d0964970 None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:27.919895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2a5a1-78b5-45d2-ad9a-a1a8d0964970 None admin] RBAC: Adding query filter params (name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:27.920203 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2a5a1-78b5-45d2-ad9a-a1a8d0964970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:27.921683 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2a5a1-78b5-45d2-ad9a-a1a8d0964970 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:27.929888 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 609/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jun 23 09:07:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:27.936731 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537acee7-0e7c-4af6-90a3-92cae6f51eb2 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:28.034393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:28.049705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537acee7-0e7c-4af6-90a3-92cae6f51eb2 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:28.081988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4fd74-a018-45bf-9f59-51c40562e4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:28.082773 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4fd74-a018-45bf-9f59-51c40562e4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:28.083106 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4fd74-a018-45bf-9f59-51c40562e4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:28.083364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4fd74-a018-45bf-9f59-51c40562e4b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:28.093447 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c4fd74-a018-45bf-9f59-51c40562e4b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:28.094885 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c4fd74-a018-45bf-9f59-51c40562e4b7 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:28.111581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-c3c4fd74-a018-45bf-9f59-51c40562e4b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:28.111581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:28.111581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:28.111581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:28.111581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:28.111581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:28.111581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:28.111581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:28.111581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:28.111581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:28.111581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:28.111581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:28.111581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:07:28.111581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:07:28.111581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:07:28.120424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:07:28.120424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:28.120424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:28.120424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:28.120424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:28.120424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:28.120424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:28.120424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:28.120424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:28.120424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:28.120424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:28.120424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:28.120424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:28.120424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:28.120424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:28.120424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:28.120424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:28.121678 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:07:28.121678 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:07:28.121678 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:28.121678 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:07:28.121678 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 628/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:07:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:28.122050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2a5a1-78b5-45d2-ad9a-a1a8d0964970 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:28.221436 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2a5a1-78b5-45d2-ad9a-a1a8d0964970 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:28.230214 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:28.255462 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce3cff-9207-4185-b4f1-5fb53c09c271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:28.256132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce3cff-9207-4185-b4f1-5fb53c09c271 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:28.256421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce3cff-9207-4185-b4f1-5fb53c09c271 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:28.256699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce3cff-9207-4185-b4f1-5fb53c09c271 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:28.257429 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce3cff-9207-4185-b4f1-5fb53c09c271 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:28.257753 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce3cff-9207-4185-b4f1-5fb53c09c271 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:28.259139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce3cff-9207-4185-b4f1-5fb53c09c271 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:28.271412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 610/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:28.277193 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4fd74-a018-45bf-9f59-51c40562e4b7 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:28.382411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4fd74-a018-45bf-9f59-51c40562e4b7 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:28.415509 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2731c19-9857-49ea-94d9-632180251128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:28.416314 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2731c19-9857-49ea-94d9-632180251128 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:28.416601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2731c19-9857-49ea-94d9-632180251128 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:28.416883 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2731c19-9857-49ea-94d9-632180251128 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:28.419575 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:28.424168 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2731c19-9857-49ea-94d9-632180251128 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:28.425695 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2731c19-9857-49ea-94d9-632180251128 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:28.440348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-f2731c19-9857-49ea-94d9-632180251128 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 09:07:28.440348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:28.440348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:28.440348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:28.440348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:28.440348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:28.440348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:28.440348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:28.440348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:28.440348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:28.440348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:28.440348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:28.440348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:07:28.440348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:07:28.440348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:07:28.440348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:07:28.441550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:28.441550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:28.441550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:07:28.441550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:07:28.441550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:07:28.441550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:07:28.441550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:28.441550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:28.441550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:28.441550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:28.441550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:28.441550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:28.441550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:28.441550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:28.441550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:28.441550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:28.441550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:28.441550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:28.442743 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:28.442743 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:28.442743 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:07:28.442743 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:07:28.442743 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:07:28.442743 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:07:28.442743 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:07:28.442743 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:07:28.442743 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 09:07:28.442743 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:07:28.442743 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 629/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:07:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:28.448121 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce3cff-9207-4185-b4f1-5fb53c09c271 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:28.546928 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce3cff-9207-4185-b4f1-5fb53c09c271 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:28.580546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814916f-e6a5-4b20-8571-6c1b3d823d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:28.581217 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814916f-e6a5-4b20-8571-6c1b3d823d29 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:28.581504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814916f-e6a5-4b20-8571-6c1b3d823d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:28.581785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814916f-e6a5-4b20-8571-6c1b3d823d29 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:28.582618 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2814916f-e6a5-4b20-8571-6c1b3d823d29 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:28.582950 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2814916f-e6a5-4b20-8571-6c1b3d823d29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:28.584388 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2814916f-e6a5-4b20-8571-6c1b3d823d29 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:28.601652 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 611/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jun 23 09:07:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:28.607997 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2731c19-9857-49ea-94d9-632180251128 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:28.706268 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2731c19-9857-49ea-94d9-632180251128 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:28.739311 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0ab1b-f57d-4338-93db-81af25128cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:28.739958 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0ab1b-f57d-4338-93db-81af25128cbd None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:28.745820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0ab1b-f57d-4338-93db-81af25128cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:28.745820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0ab1b-f57d-4338-93db-81af25128cbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:28.798990 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:28.805334 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0ab1b-f57d-4338-93db-81af25128cbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:28.807578 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0ab1b-f57d-4338-93db-81af25128cbd None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:28.904074 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 630/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:07:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 09:07:28.909970 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814916f-e6a5-4b20-8571-6c1b3d823d29 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:28.936703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:29.005335 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814916f-e6a5-4b20-8571-6c1b3d823d29 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:29.038808 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245aa74-5f99-4a95-b85b-9a54f9aa5fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:29.039612 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245aa74-5f99-4a95-b85b-9a54f9aa5fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:29.039893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245aa74-5f99-4a95-b85b-9a54f9aa5fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:29.040163 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245aa74-5f99-4a95-b85b-9a54f9aa5fb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:29.050366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a245aa74-5f99-4a95-b85b-9a54f9aa5fb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:29.051835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a245aa74-5f99-4a95-b85b-9a54f9aa5fb6 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:29.075053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-a245aa74-5f99-4a95-b85b-9a54f9aa5fb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:29.075053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:29.075053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:29.075053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:29.075053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:29.075053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:29.075053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:29.075053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:29.075053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:29.075053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:29.075053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:29.075053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:29.075053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:07:29.075053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:07:29.075053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:07:29.076293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:07:29.076293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:29.076293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:29.076293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:29.076293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:29.076293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:29.076293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:29.076293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:29.076293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:29.076293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:29.076293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:29.076293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:29.076293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:29.076293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:29.076293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:29.076293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:29.076293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:29.078663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:07:29.078663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:07:29.078663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:29.078663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:29.078663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 612/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:07:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:29.083347 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0ab1b-f57d-4338-93db-81af25128cbd None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:29.174156 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:29.179798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0ab1b-f57d-4338-93db-81af25128cbd None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:29.215769 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4aec2a-8708-44b2-a1ec-5938ec1c5aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:29.216428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4aec2a-8708-44b2-a1ec-5938ec1c5aed None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:29.216712 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4aec2a-8708-44b2-a1ec-5938ec1c5aed None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:29.216987 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4aec2a-8708-44b2-a1ec-5938ec1c5aed None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:29.217734 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4aec2a-8708-44b2-a1ec-5938ec1c5aed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:29.218039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4aec2a-8708-44b2-a1ec-5938ec1c5aed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:29.219454 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4aec2a-8708-44b2-a1ec-5938ec1c5aed None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:29.230806 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 631/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:07:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:29.236387 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245aa74-5f99-4a95-b85b-9a54f9aa5fb6 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:29.313375 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:29.334126 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245aa74-5f99-4a95-b85b-9a54f9aa5fb6 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:29.372819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa080f7c-2d90-47bb-8429-268897508040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:29.373643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa080f7c-2d90-47bb-8429-268897508040 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:29.373928 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa080f7c-2d90-47bb-8429-268897508040 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:29.374205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa080f7c-2d90-47bb-8429-268897508040 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:29.381093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa080f7c-2d90-47bb-8429-268897508040 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:29.382527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa080f7c-2d90-47bb-8429-268897508040 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:29.395814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-fa080f7c-2d90-47bb-8429-268897508040 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:07:29.395814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:29.395814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:29.395814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:29.395814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:29.395814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:29.395814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:29.395814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:29.395814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:29.395814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:29.395814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:29.395814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:29.395814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:07:29.395814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:07:29.395814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:07:29.396898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:07:29.396898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:29.396898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:29.396898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:29.396898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:29.396898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:29.396898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:29.396898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:29.396898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:29.396898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:29.396898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:29.396898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:29.396898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:29.396898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:29.396898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:29.396898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:29.396898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:29.398329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:07:29.398329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:07:29.398329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:07:29.398329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:07:29.398329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:07:29.398329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:07:29.398329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:07:29.398329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:29.398329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 613/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:07:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:29.402934 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4aec2a-8708-44b2-a1ec-5938ec1c5aed None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:29.499137 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4aec2a-8708-44b2-a1ec-5938ec1c5aed None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:29.530459 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824a58c-e578-403c-bf19-2cda8134a607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:29.531159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824a58c-e578-403c-bf19-2cda8134a607 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:29.531444 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824a58c-e578-403c-bf19-2cda8134a607 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:29.531719 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824a58c-e578-403c-bf19-2cda8134a607 None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:29.532548 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824a58c-e578-403c-bf19-2cda8134a607 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:29.532853 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824a58c-e578-403c-bf19-2cda8134a607 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:29.534269 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824a58c-e578-403c-bf19-2cda8134a607 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:29.546668 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 632/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:07:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:29.552806 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa080f7c-2d90-47bb-8429-268897508040 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:29.656536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa080f7c-2d90-47bb-8429-268897508040 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:29.689300 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6985661-21d9-417c-b8a8-20929e472527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:29.689955 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6985661-21d9-417c-b8a8-20929e472527 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:29.690242 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6985661-21d9-417c-b8a8-20929e472527 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:29.690520 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6985661-21d9-417c-b8a8-20929e472527 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:29.691444 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6985661-21d9-417c-b8a8-20929e472527 None admin] RBAC: Adding query filter params (role.id=edf696ae763147cab68434524d1eff9c, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=98a63b20ceca40e6bceb56eda8ae9686) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:29.691757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6985661-21d9-417c-b8a8-20929e472527 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:29.693197 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6985661-21d9-417c-b8a8-20929e472527 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:29.699611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 614/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:07:29 2022] GET /identity/v3/role_assignments?user.id=98a63b20ceca40e6bceb56eda8ae9686&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=edf696ae763147cab68434524d1eff9c => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:31.765692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f8860-daa6-4e5d-b028-1a50aeafa237 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:31.766247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f8860-daa6-4e5d-b028-1a50aeafa237 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:31.766247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f8860-daa6-4e5d-b028-1a50aeafa237 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:31.767634 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 633/1247] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:07:31.772811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66ff7c-b51c-4b5b-9cef-7a8aa765699f None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:31.773202 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66ff7c-b51c-4b5b-9cef-7a8aa765699f None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:31.773397 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66ff7c-b51c-4b5b-9cef-7a8aa765699f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:31.778904 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:31.790772 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-ae66ff7c-b51c-4b5b-9cef-7a8aa765699f None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:07:31.792569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ae66ff7c-b51c-4b5b-9cef-7a8aa765699f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:07:31.857037 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 615/1248] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:07:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:07:31.880411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f8860-daa6-4e5d-b028-1a50aeafa237 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:31.897873 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:31.981431 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f8860-daa6-4e5d-b028-1a50aeafa237 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:32.014505 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba953e06-ef5e-4039-98d4-4f3b1bd9c0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:32.015319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba953e06-ef5e-4039-98d4-4f3b1bd9c0be None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:32.015606 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba953e06-ef5e-4039-98d4-4f3b1bd9c0be None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:32.015936 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba953e06-ef5e-4039-98d4-4f3b1bd9c0be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:32.025989 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba953e06-ef5e-4039-98d4-4f3b1bd9c0be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:07:32.033669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-ba953e06-ef5e-4039-98d4-4f3b1bd9c0be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:07:32.033669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:32.033669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:32.033669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:32.033669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:32.033669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:32.033669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:32.033669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:32.033669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:32.033669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:32.033669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:32.033669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:32.033669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:07:32.033669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:07:32.033669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:07:32.033669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:07:32.035341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:07:32.035341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:07:32.035341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:07:32.035341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:07:32.035341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:32.035341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:32.035341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:32.035341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:32.035341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:32.035341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:32.035341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:32.035341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:32.035341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:32.035341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:32.035341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:32.035341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:32.036293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:32.036293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:32.036293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:32.036293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:32.036293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:07:32.036293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:07:32.036293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:07:32.036293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:07:32.036293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:07:32.036293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:07:32.036293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:07:32.036293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:07:32.036293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 634/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:07:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:32.041016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66ff7c-b51c-4b5b-9cef-7a8aa765699f None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:32.138635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66ff7c-b51c-4b5b-9cef-7a8aa765699f None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:32.167009 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:32.174659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda7dcb-a4f0-4ac0-8958-8f2cf0648e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:32.175389 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda7dcb-a4f0-4ac0-8958-8f2cf0648e5d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:32.175706 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda7dcb-a4f0-4ac0-8958-8f2cf0648e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:32.175995 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda7dcb-a4f0-4ac0-8958-8f2cf0648e5d None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:32.176739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda7dcb-a4f0-4ac0-8958-8f2cf0648e5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:32.177043 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda7dcb-a4f0-4ac0-8958-8f2cf0648e5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:32.178410 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda7dcb-a4f0-4ac0-8958-8f2cf0648e5d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:32.187191 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 616/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:07:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:32.193107 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba953e06-ef5e-4039-98d4-4f3b1bd9c0be None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:32.269224 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:32.291203 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba953e06-ef5e-4039-98d4-4f3b1bd9c0be None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:32.322992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d6d68-d7db-44c4-905a-1adbf25191f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:32.323818 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d6d68-d7db-44c4-905a-1adbf25191f8 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:32.324104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d6d68-d7db-44c4-905a-1adbf25191f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:32.324380 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d6d68-d7db-44c4-905a-1adbf25191f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:32.334894 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d6d68-d7db-44c4-905a-1adbf25191f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:32.336368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d6d68-d7db-44c4-905a-1adbf25191f8 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:32.353477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-b90d6d68-d7db-44c4-905a-1adbf25191f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:32.353477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:32.353477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:32.353477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:32.353477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:32.353477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:32.353477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:32.353477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:32.353477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:32.353477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:32.353477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:32.353477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:32.353477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:07:32.353477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:07:32.353477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:07:32.354599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:07:32.354599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:32.354599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:32.354599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:32.354599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:32.354599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:32.354599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:32.354599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:32.354599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:32.354599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:32.354599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:32.354599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:32.354599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:32.354599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:32.354599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:32.354599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:32.354599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:32.356095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:07:32.356095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:07:32.356095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:32.356095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:07:32.356095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 635/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:32.360419 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda7dcb-a4f0-4ac0-8958-8f2cf0648e5d None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:32.459133 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda7dcb-a4f0-4ac0-8958-8f2cf0648e5d None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:32.492305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4164b-d68d-47b9-b4ff-b704a455c873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:32.492964 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4164b-d68d-47b9-b4ff-b704a455c873 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:32.493257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4164b-d68d-47b9-b4ff-b704a455c873 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:32.493538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4164b-d68d-47b9-b4ff-b704a455c873 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:32.494289 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4164b-d68d-47b9-b4ff-b704a455c873 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:32.494601 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4164b-d68d-47b9-b4ff-b704a455c873 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:32.496015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4164b-d68d-47b9-b4ff-b704a455c873 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:32.508484 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 617/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:32.514718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d6d68-d7db-44c4-905a-1adbf25191f8 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:32.614162 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d6d68-d7db-44c4-905a-1adbf25191f8 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:32.640824 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:32.648488 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49305915-afa7-4afa-9eb4-db0a6fa5268f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:32.649283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49305915-afa7-4afa-9eb4-db0a6fa5268f None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:32.649573 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49305915-afa7-4afa-9eb4-db0a6fa5268f None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:32.654208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49305915-afa7-4afa-9eb4-db0a6fa5268f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:32.657955 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49305915-afa7-4afa-9eb4-db0a6fa5268f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:32.659436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49305915-afa7-4afa-9eb4-db0a6fa5268f None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:32.674262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-49305915-afa7-4afa-9eb4-db0a6fa5268f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 09:07:32.674262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:32.674262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:32.674262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:32.674262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:32.674262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:32.674262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:32.674262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:32.674262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:32.674262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:32.674262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:32.674262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:32.674262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:07:32.674262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:07:32.674262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:07:32.674262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:07:32.677982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:32.677982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:32.677982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:07:32.677982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:07:32.677982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:07:32.677982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:07:32.677982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:32.677982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:32.677982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:32.677982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:32.677982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:32.677982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:32.677982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:32.677982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:32.677982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:32.677982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:32.677982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:32.677982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:32.679269 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:32.679269 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:32.679269 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:07:32.679269 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:07:32.679269 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:07:32.679269 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:07:32.679269 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:07:32.679269 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:07:32.679269 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 09:07:32.679269 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:07:32.679269 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 636/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:07:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:32.681455 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4164b-d68d-47b9-b4ff-b704a455c873 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:32.737064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:32.780274 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4164b-d68d-47b9-b4ff-b704a455c873 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:32.813339 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab8b59-5165-4020-b23e-7c72f0a69ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:32.814009 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab8b59-5165-4020-b23e-7c72f0a69ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:32.814296 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab8b59-5165-4020-b23e-7c72f0a69ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:32.814575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab8b59-5165-4020-b23e-7c72f0a69ba1 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:32.815456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab8b59-5165-4020-b23e-7c72f0a69ba1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:32.815773 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab8b59-5165-4020-b23e-7c72f0a69ba1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:32.817223 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab8b59-5165-4020-b23e-7c72f0a69ba1 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:32.833778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 618/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jun 23 09:07:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:07:32.840364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49305915-afa7-4afa-9eb4-db0a6fa5268f None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:32.939227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49305915-afa7-4afa-9eb4-db0a6fa5268f None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:32.972260 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972850f-ab12-43c8-9848-b681b04da402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:32.972905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972850f-ab12-43c8-9848-b681b04da402 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:32.973202 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972850f-ab12-43c8-9848-b681b04da402 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:32.973457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972850f-ab12-43c8-9848-b681b04da402 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:33.026937 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:33.038787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972850f-ab12-43c8-9848-b681b04da402 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:33.040266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972850f-ab12-43c8-9848-b681b04da402 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:33.142477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 637/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:07:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 09:07:33.146331 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab8b59-5165-4020-b23e-7c72f0a69ba1 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:33.244043 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab8b59-5165-4020-b23e-7c72f0a69ba1 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:33.256668 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:33.277892 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a725fd-9d0f-46a5-a89f-fde041b484a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:33.279829 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a725fd-9d0f-46a5-a89f-fde041b484a7 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:33.279829 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a725fd-9d0f-46a5-a89f-fde041b484a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:33.279829 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a725fd-9d0f-46a5-a89f-fde041b484a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:33.289541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a725fd-9d0f-46a5-a89f-fde041b484a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:33.291080 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a725fd-9d0f-46a5-a89f-fde041b484a7 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:33.309038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-73a725fd-9d0f-46a5-a89f-fde041b484a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:33.309038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:33.309038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:33.309038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:33.309038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:33.309038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:33.309038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:33.309038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:33.309038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:33.309038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:33.309038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:33.309038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:33.309038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:07:33.309038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:07:33.309038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:07:33.310147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:07:33.310147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:33.310147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:33.310147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:33.310147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:33.310147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:33.310147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:33.310147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:33.310147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:33.310147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:33.310147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:33.310147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:33.310147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:33.310147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:33.310147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:33.310147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:33.310147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:33.311469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:07:33.311469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:07:33.311469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:33.311469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:33.311469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 619/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:07:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:33.316086 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972850f-ab12-43c8-9848-b681b04da402 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:33.401567 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:33.412973 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972850f-ab12-43c8-9848-b681b04da402 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:33.446165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184f975-f16f-4f3d-bcf5-032334005453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:33.446832 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184f975-f16f-4f3d-bcf5-032334005453 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:33.447161 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184f975-f16f-4f3d-bcf5-032334005453 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:33.447990 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184f975-f16f-4f3d-bcf5-032334005453 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:33.448722 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2184f975-f16f-4f3d-bcf5-032334005453 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:33.449062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2184f975-f16f-4f3d-bcf5-032334005453 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:33.450437 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2184f975-f16f-4f3d-bcf5-032334005453 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:33.462225 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 638/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:33.468335 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a725fd-9d0f-46a5-a89f-fde041b484a7 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:33.565996 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a725fd-9d0f-46a5-a89f-fde041b484a7 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:33.599126 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf7c5e-683e-4d58-89d5-54d0777bb365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:33.599907 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf7c5e-683e-4d58-89d5-54d0777bb365 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:33.600196 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf7c5e-683e-4d58-89d5-54d0777bb365 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:33.600476 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf7c5e-683e-4d58-89d5-54d0777bb365 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:33.607042 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf7c5e-683e-4d58-89d5-54d0777bb365 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:33.608494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf7c5e-683e-4d58-89d5-54d0777bb365 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:33.622546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-42bf7c5e-683e-4d58-89d5-54d0777bb365 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:07:33.622546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:33.622546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:33.622546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:33.622546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:33.622546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:33.622546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:33.622546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:33.622546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:33.622546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:33.622546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:33.622546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:33.622546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:07:33.622546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:07:33.622546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:07:33.628483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:07:33.628483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:33.628483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:33.628483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:33.628483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:33.628483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:33.628483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:33.628483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:33.628483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:33.628483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:33.628483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:33.628483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:33.628483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:33.628483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:33.628483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:33.628483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:33.628483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:33.629508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:07:33.629508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:07:33.629508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:07:33.629508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:07:33.629508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:07:33.629508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:07:33.629508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:07:33.629508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:33.629508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 620/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:07:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:33.630073 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184f975-f16f-4f3d-bcf5-032334005453 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:33.728011 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184f975-f16f-4f3d-bcf5-032334005453 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:33.760888 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd5859-2faa-4537-8448-f977a3220391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:33.761550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd5859-2faa-4537-8448-f977a3220391 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:33.761854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd5859-2faa-4537-8448-f977a3220391 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:33.762120 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd5859-2faa-4537-8448-f977a3220391 None admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:33.762942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd5859-2faa-4537-8448-f977a3220391 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:33.763260 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd5859-2faa-4537-8448-f977a3220391 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:33.764757 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd5859-2faa-4537-8448-f977a3220391 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:33.777029 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 639/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:07:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:33.783141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf7c5e-683e-4d58-89d5-54d0777bb365 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:33.809514 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:33.881218 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf7c5e-683e-4d58-89d5-54d0777bb365 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:33.915366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438e909-f9ff-45e8-881f-77465aab52ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:33.916019 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438e909-f9ff-45e8-881f-77465aab52ba None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:33.916305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438e909-f9ff-45e8-881f-77465aab52ba None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:33.923813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438e909-f9ff-45e8-881f-77465aab52ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:33.923813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438e909-f9ff-45e8-881f-77465aab52ba None admin] RBAC: Adding query filter params (role.id=9cd080b50c6f4ee8a54bf9ef1655bd83, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=98a63b20ceca40e6bceb56eda8ae9686) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:33.923813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438e909-f9ff-45e8-881f-77465aab52ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:33.923813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438e909-f9ff-45e8-881f-77465aab52ba None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:33.924471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 621/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:07:33 2022] GET /identity/v3/role_assignments?user.id=98a63b20ceca40e6bceb56eda8ae9686&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:35.953134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75081f9-2810-458b-ab52-7c799382fc99 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:35.953672 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75081f9-2810-458b-ab52-7c799382fc99 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:35.953770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75081f9-2810-458b-ab52-7c799382fc99 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:35.955116 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 640/1261] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:07:35.960221 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e661fc-7916-4a39-a155-d6a21e217409 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:35.960624 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e661fc-7916-4a39-a155-d6a21e217409 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:35.960781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e661fc-7916-4a39-a155-d6a21e217409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:35.977958 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-66e661fc-7916-4a39-a155-d6a21e217409 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:07:35.979720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-66e661fc-7916-4a39-a155-d6a21e217409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:07:36.044012 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 622/1262] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:07:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:07:36.065947 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75081f9-2810-458b-ab52-7c799382fc99 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:36.077215 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:36.165323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75081f9-2810-458b-ab52-7c799382fc99 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:36.198547 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0134c-8b02-4935-8147-6e8a63db6fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:36.199354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0134c-8b02-4935-8147-6e8a63db6fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:36.199637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0134c-8b02-4935-8147-6e8a63db6fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:36.199912 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0134c-8b02-4935-8147-6e8a63db6fd0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:36.210371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0134c-8b02-4935-8147-6e8a63db6fd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:07:36.219117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-b1e0134c-8b02-4935-8147-6e8a63db6fd0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:07:36.219117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:36.219117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:36.219117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:36.219117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:36.219117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:36.219117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:36.219117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:36.219117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:36.219117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:36.219117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:36.219117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:36.219117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:07:36.219117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:07:36.219117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:07:36.219117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:07:36.221939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:07:36.221939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:07:36.221939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:07:36.221939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:07:36.221939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:36.221939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:36.221939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:36.221939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:36.221939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:36.221939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:36.221939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:36.221939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:36.221939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:36.221939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:36.221939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:36.221939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:36.223088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:36.223088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:36.223088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:36.223088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:36.223088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:07:36.223088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:07:36.223088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:07:36.223088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:07:36.223088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:07:36.223088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:07:36.223088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:07:36.223088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:07:36.223088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 641/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:07:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:36.226482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e661fc-7916-4a39-a155-d6a21e217409 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:36.245716 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:36.322990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e661fc-7916-4a39-a155-d6a21e217409 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:36.355856 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba276187-f06e-43e8-934c-0f29aa23fd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:36.356522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba276187-f06e-43e8-934c-0f29aa23fd4d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:36.356804 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba276187-f06e-43e8-934c-0f29aa23fd4d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:36.357080 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba276187-f06e-43e8-934c-0f29aa23fd4d None admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:36.357890 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba276187-f06e-43e8-934c-0f29aa23fd4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:36.358199 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba276187-f06e-43e8-934c-0f29aa23fd4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:36.359636 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba276187-f06e-43e8-934c-0f29aa23fd4d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:36.367367 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 623/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:07:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:36.373698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0134c-8b02-4935-8147-6e8a63db6fd0 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:36.440588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:36.473103 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0134c-8b02-4935-8147-6e8a63db6fd0 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:36.505073 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a3965-f690-4b68-97d7-a23c3cfba496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:36.505933 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a3965-f690-4b68-97d7-a23c3cfba496 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:36.506226 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a3965-f690-4b68-97d7-a23c3cfba496 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:36.506508 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a3965-f690-4b68-97d7-a23c3cfba496 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:36.517014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a3965-f690-4b68-97d7-a23c3cfba496 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:36.518527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a3965-f690-4b68-97d7-a23c3cfba496 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:36.535988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-a12a3965-f690-4b68-97d7-a23c3cfba496 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:36.535988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:36.535988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:36.535988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:36.535988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:36.535988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:36.535988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:36.535988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:36.535988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:36.535988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:36.535988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:36.535988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:36.535988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:07:36.535988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:07:36.535988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:07:36.537171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:07:36.537171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:36.537171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:36.537171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:36.537171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:36.537171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:36.537171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:36.537171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:36.537171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:36.537171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:36.537171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:36.537171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:36.537171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:36.537171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:36.537171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:36.537171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:36.537171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:36.538368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:07:36.538368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:07:36.538368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:36.538368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:07:36.538368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 642/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:07:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:36.543572 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba276187-f06e-43e8-934c-0f29aa23fd4d None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:36.627211 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:36.645724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba276187-f06e-43e8-934c-0f29aa23fd4d None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:36.678674 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624f61c-f005-4eab-bfdc-b5344f9610df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:36.679371 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624f61c-f005-4eab-bfdc-b5344f9610df None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:36.679661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624f61c-f005-4eab-bfdc-b5344f9610df None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:36.679940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624f61c-f005-4eab-bfdc-b5344f9610df None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:36.680666 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c624f61c-f005-4eab-bfdc-b5344f9610df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:36.680974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c624f61c-f005-4eab-bfdc-b5344f9610df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:36.682857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c624f61c-f005-4eab-bfdc-b5344f9610df None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:36.694990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 624/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:36.700811 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a3965-f690-4b68-97d7-a23c3cfba496 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:36.798745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a3965-f690-4b68-97d7-a23c3cfba496 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:36.817318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:36.830760 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42ba29-cf4e-4fdd-8353-8fd4967a319c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:36.831617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42ba29-cf4e-4fdd-8353-8fd4967a319c None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:36.831901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42ba29-cf4e-4fdd-8353-8fd4967a319c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:36.832176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42ba29-cf4e-4fdd-8353-8fd4967a319c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:36.840053 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c42ba29-cf4e-4fdd-8353-8fd4967a319c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:36.841483 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c42ba29-cf4e-4fdd-8353-8fd4967a319c None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:36.855175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-7c42ba29-cf4e-4fdd-8353-8fd4967a319c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 09:07:36.855175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:36.855175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:36.855175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:36.855175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:36.855175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:36.855175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:36.855175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:36.855175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:36.855175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:36.855175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:36.855175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:36.855175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:07:36.855175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:07:36.855175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:07:36.855175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:07:36.856359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:36.856359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:36.856359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:07:36.856359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:07:36.856359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:07:36.856359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:07:36.856359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:36.856359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:36.856359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:36.856359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:36.856359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:36.856359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:36.856359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:36.856359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:36.856359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:36.856359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:36.856359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:36.856359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:36.857619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:36.857619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:36.857619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:07:36.857619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:07:36.857619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:07:36.857619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:07:36.857619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:07:36.857619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:07:36.857619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 09:07:36.857619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:07:36.857619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 643/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:07:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:36.862237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624f61c-f005-4eab-bfdc-b5344f9610df None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:36.961666 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624f61c-f005-4eab-bfdc-b5344f9610df None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:36.996780 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9b037-99e8-4b79-b9b5-745dd2c58c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:36.997592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9b037-99e8-4b79-b9b5-745dd2c58c47 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:36.997878 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9b037-99e8-4b79-b9b5-745dd2c58c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:36.998152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9b037-99e8-4b79-b9b5-745dd2c58c47 None admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:36.999006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9b037-99e8-4b79-b9b5-745dd2c58c47 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:36.999317 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9b037-99e8-4b79-b9b5-745dd2c58c47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:37.000744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9b037-99e8-4b79-b9b5-745dd2c58c47 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:37.009269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:37.017980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 625/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jun 23 09:07:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:37.023672 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42ba29-cf4e-4fdd-8353-8fd4967a319c None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:37.121539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42ba29-cf4e-4fdd-8353-8fd4967a319c None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:37.156257 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832d53c-f46e-46ea-8b4d-734ad0df816a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:37.157082 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832d53c-f46e-46ea-8b4d-734ad0df816a None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:37.157370 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832d53c-f46e-46ea-8b4d-734ad0df816a None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:37.157647 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832d53c-f46e-46ea-8b4d-734ad0df816a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:37.168128 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832d53c-f46e-46ea-8b4d-734ad0df816a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:37.170804 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832d53c-f46e-46ea-8b4d-734ad0df816a None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:37.191617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-7832d53c-f46e-46ea-8b4d-734ad0df816a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:37.191617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:37.191617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:37.191617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:37.191617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:37.191617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:37.191617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:37.191617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:37.191617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:37.191617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:37.191617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:37.191617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:37.191617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:07:37.191617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:07:37.191617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:07:37.192815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:07:37.192815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:37.192815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:37.192815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:37.192815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:37.192815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:37.192815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:37.192815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:37.192815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:37.192815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:37.192815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:37.192815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:37.192815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:37.192815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:37.192815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:37.192815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:37.192815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:37.193996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:07:37.193996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:07:37.193996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:37.193996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:07:37.193996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 644/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:07:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:37.198735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9b037-99e8-4b79-b9b5-745dd2c58c47 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:37.297351 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9b037-99e8-4b79-b9b5-745dd2c58c47 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:37.330974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d453cb8-1358-4663-a3bd-86c09155f665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:37.331691 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d453cb8-1358-4663-a3bd-86c09155f665 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:37.331982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d453cb8-1358-4663-a3bd-86c09155f665 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:37.332261 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d453cb8-1358-4663-a3bd-86c09155f665 None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:37.332983 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d453cb8-1358-4663-a3bd-86c09155f665 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:37.333290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d453cb8-1358-4663-a3bd-86c09155f665 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:37.334796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d453cb8-1358-4663-a3bd-86c09155f665 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:37.346198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 626/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:37.351890 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832d53c-f46e-46ea-8b4d-734ad0df816a None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:37.383222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:37.452369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832d53c-f46e-46ea-8b4d-734ad0df816a None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:37.485598 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ce9c4-b86d-4873-a4f0-1a5d40b633a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:37.486415 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ce9c4-b86d-4873-a4f0-1a5d40b633a8 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:37.486697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ce9c4-b86d-4873-a4f0-1a5d40b633a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:37.486997 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ce9c4-b86d-4873-a4f0-1a5d40b633a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:37.493292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ce9c4-b86d-4873-a4f0-1a5d40b633a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:37.495383 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ce9c4-b86d-4873-a4f0-1a5d40b633a8 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:37.509397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-943ce9c4-b86d-4873-a4f0-1a5d40b633a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:07:37.509397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:37.509397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:37.509397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:37.509397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:37.509397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:37.509397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:37.509397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:37.509397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:37.509397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:37.509397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:37.509397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:37.509397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:07:37.509397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:07:37.509397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:07:37.512907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:07:37.512907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:37.512907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:37.512907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:37.512907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:37.512907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:37.512907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:37.512907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:37.512907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:37.512907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:37.512907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:37.512907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:37.512907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:37.512907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:37.512907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:37.512907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:37.512907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:37.514167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:07:37.514167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:07:37.514167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:07:37.514167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:07:37.514167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:07:37.514167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:07:37.514167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:07:37.514167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:07:37.514167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 645/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:07:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:37.516697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d453cb8-1358-4663-a3bd-86c09155f665 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:37.575401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:37.620169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d453cb8-1358-4663-a3bd-86c09155f665 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:37.653358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dec22a-162a-4a56-8d3f-1c2adca2c8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:37.654435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dec22a-162a-4a56-8d3f-1c2adca2c8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:37.654962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dec22a-162a-4a56-8d3f-1c2adca2c8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:37.655457 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dec22a-162a-4a56-8d3f-1c2adca2c8a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:37.656933 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dec22a-162a-4a56-8d3f-1c2adca2c8a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:37.657491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dec22a-162a-4a56-8d3f-1c2adca2c8a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:37.659389 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dec22a-162a-4a56-8d3f-1c2adca2c8a9 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:37.671994 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 627/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:07:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:37.677948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ce9c4-b86d-4873-a4f0-1a5d40b633a8 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:37.778799 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:37.787298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ce9c4-b86d-4873-a4f0-1a5d40b633a8 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:37.820731 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a17ee-568c-45e3-9014-29ab951d9640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:37.821613 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a17ee-568c-45e3-9014-29ab951d9640 None admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:37.821898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a17ee-568c-45e3-9014-29ab951d9640 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:37.822169 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a17ee-568c-45e3-9014-29ab951d9640 None admin] PATH_INFO: `/v3/projects/93b5040533524222b126bbe93afc56d1/users/98a63b20ceca40e6bceb56eda8ae9686/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:37.824131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a17ee-568c-45e3-9014-29ab951d9640 None admin] RBAC: Authorizing `identity:create_grant(project_id=93b5040533524222b126bbe93afc56d1, user_id=98a63b20ceca40e6bceb56eda8ae9686, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:37.825552 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a17ee-568c-45e3-9014-29ab951d9640 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:38.096140 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 646/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jun 23 09:07:37 2022] PUT /identity/v3/projects/93b5040533524222b126bbe93afc56d1/users/98a63b20ceca40e6bceb56eda8ae9686/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:07:40.111976 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1eb986-e8e8-454d-9ae2-5267be17ae0a None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:40.112614 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1eb986-e8e8-454d-9ae2-5267be17ae0a None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:40.112614 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1eb986-e8e8-454d-9ae2-5267be17ae0a None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:40.113961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 628/1274] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:07:40.119812 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c6b69-73c9-408a-b386-92940aa07f54 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:40.120210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c6b69-73c9-408a-b386-92940aa07f54 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:40.120379 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c6b69-73c9-408a-b386-92940aa07f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:40.137863 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-665c6b69-73c9-408a-b386-92940aa07f54 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:07:40.139085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-665c6b69-73c9-408a-b386-92940aa07f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:07:40.207112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 647/1275] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:07:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:07:40.231751 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1eb986-e8e8-454d-9ae2-5267be17ae0a None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:40.330581 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1eb986-e8e8-454d-9ae2-5267be17ae0a None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:40.348274 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:40.365336 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96443c14-70d6-4a91-974e-514448c9fcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:40.366055 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96443c14-70d6-4a91-974e-514448c9fcff None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:40.366336 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96443c14-70d6-4a91-974e-514448c9fcff None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:40.366609 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96443c14-70d6-4a91-974e-514448c9fcff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:40.376731 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96443c14-70d6-4a91-974e-514448c9fcff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 09:07:40.384310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-96443c14-70d6-4a91-974e-514448c9fcff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:07:40.384310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:40.384310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:40.384310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:40.384310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:40.384310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:40.384310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:40.384310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:40.384310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:40.384310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:40.384310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:40.384310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:40.384310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 09:07:40.384310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 09:07:40.384310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 09:07:40.384310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 09:07:40.385675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 09:07:40.385675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 09:07:40.385675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 09:07:40.385675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 09:07:40.385675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:40.385675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:40.385675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:40.385675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:40.385675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:40.385675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:40.385675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:40.385675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:40.385675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:40.385675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:40.385675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:40.385675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:40.386873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:40.386873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:40.386873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:40.386873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:40.386873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 09:07:40.386873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 09:07:40.386873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 09:07:40.386873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 09:07:40.386873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 09:07:40.386873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 09:07:40.386873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 09:07:40.386873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:40.386873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 629/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jun 23 09:07:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:40.393764 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c6b69-73c9-408a-b386-92940aa07f54 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:40.473473 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:40.490970 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c6b69-73c9-408a-b386-92940aa07f54 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:40.523847 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889162f-5dbf-4167-85fb-a7892a3e912f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:40.524516 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889162f-5dbf-4167-85fb-a7892a3e912f None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:40.524804 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889162f-5dbf-4167-85fb-a7892a3e912f None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:40.525091 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889162f-5dbf-4167-85fb-a7892a3e912f None admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:40.526424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889162f-5dbf-4167-85fb-a7892a3e912f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:40.526732 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889162f-5dbf-4167-85fb-a7892a3e912f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:40.528140 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889162f-5dbf-4167-85fb-a7892a3e912f None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:40.536050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 648/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jun 23 09:07:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:40.541857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96443c14-70d6-4a91-974e-514448c9fcff None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:40.638814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96443c14-70d6-4a91-974e-514448c9fcff None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:40.671097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04348a4c-61e7-4718-ab3c-4d7447deae3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:40.683192 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04348a4c-61e7-4718-ab3c-4d7447deae3d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:40.683192 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04348a4c-61e7-4718-ab3c-4d7447deae3d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:40.683192 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04348a4c-61e7-4718-ab3c-4d7447deae3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:40.683192 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04348a4c-61e7-4718-ab3c-4d7447deae3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:40.683978 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04348a4c-61e7-4718-ab3c-4d7447deae3d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:40.701083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-04348a4c-61e7-4718-ab3c-4d7447deae3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:40.701083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:40.701083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:40.701083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:40.701083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:40.701083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:40.701083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:40.701083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:40.701083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:40.701083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:40.701083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:40.701083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:40.701083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:07:40.701083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:07:40.701083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:07:40.702227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:07:40.702227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:40.702227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:40.702227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:40.702227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:40.702227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:40.702227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:40.702227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:40.702227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:40.702227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:40.702227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:40.702227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:40.702227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:40.702227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:40.702227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:40.702227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:40.702227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:40.703273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:07:40.703273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:07:40.703273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:40.703273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:40.703273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 630/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:40.708118 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889162f-5dbf-4167-85fb-a7892a3e912f None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:40.804185 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889162f-5dbf-4167-85fb-a7892a3e912f None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:40.836222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2c6b9-1dab-4643-93bf-cfc6502aa3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:40.836892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2c6b9-1dab-4643-93bf-cfc6502aa3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:40.837229 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2c6b9-1dab-4643-93bf-cfc6502aa3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:40.837507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2c6b9-1dab-4643-93bf-cfc6502aa3d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:40.839213 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2c6b9-1dab-4643-93bf-cfc6502aa3d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:40.839531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2c6b9-1dab-4643-93bf-cfc6502aa3d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:40.840946 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2c6b9-1dab-4643-93bf-cfc6502aa3d5 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:40.852377 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 649/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:40.858045 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04348a4c-61e7-4718-ab3c-4d7447deae3d None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:40.889600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:40.955960 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04348a4c-61e7-4718-ab3c-4d7447deae3d None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:40.989166 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc08a8-a004-4805-a6b1-5ba0f2de1c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:40.989972 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc08a8-a004-4805-a6b1-5ba0f2de1c0c None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:40.990259 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc08a8-a004-4805-a6b1-5ba0f2de1c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:40.990539 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc08a8-a004-4805-a6b1-5ba0f2de1c0c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:40.998721 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc08a8-a004-4805-a6b1-5ba0f2de1c0c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:41.000164 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc08a8-a004-4805-a6b1-5ba0f2de1c0c None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:41.014589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-c1dc08a8-a004-4805-a6b1-5ba0f2de1c0c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 09:07:41.014589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:41.014589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:41.014589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:41.014589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:41.014589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:41.014589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:41.014589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:41.014589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:41.014589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:41.014589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:41.014589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:41.014589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 09:07:41.014589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 09:07:41.014589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 09:07:41.014589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 09:07:41.015796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:41.015796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:41.015796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 09:07:41.015796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:07:41.015796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 09:07:41.015796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 09:07:41.015796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:41.015796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:41.015796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:41.015796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:41.015796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:41.015796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:41.015796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:41.015796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:41.015796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:41.015796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:41.015796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:41.015796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:41.017198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:41.017198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:41.017198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 09:07:41.017198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 09:07:41.017198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 09:07:41.017198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 09:07:41.017198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 09:07:41.017198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 09:07:41.017198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 09:07:41.017198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:41.017198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 631/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:07:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:41.021662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2c6b9-1dab-4643-93bf-cfc6502aa3d5 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:41.025393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:41.118193 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2c6b9-1dab-4643-93bf-cfc6502aa3d5 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:41.152085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310763a-bdbf-4e37-90d0-8468aa91836c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:41.152684 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310763a-bdbf-4e37-90d0-8468aa91836c None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:41.152978 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310763a-bdbf-4e37-90d0-8468aa91836c None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:41.153258 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310763a-bdbf-4e37-90d0-8468aa91836c None admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:41.154100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c310763a-bdbf-4e37-90d0-8468aa91836c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:41.154411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c310763a-bdbf-4e37-90d0-8468aa91836c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:41.155874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c310763a-bdbf-4e37-90d0-8468aa91836c None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:41.172910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 650/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jun 23 09:07:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:41.182235 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc08a8-a004-4805-a6b1-5ba0f2de1c0c None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:41.273850 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:41.283580 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc08a8-a004-4805-a6b1-5ba0f2de1c0c None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:41.316426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1da5b4-e5d9-4a86-845e-8ce808a3672c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:41.317077 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1da5b4-e5d9-4a86-845e-8ce808a3672c None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:41.317363 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1da5b4-e5d9-4a86-845e-8ce808a3672c None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:41.317643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1da5b4-e5d9-4a86-845e-8ce808a3672c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:41.382578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1da5b4-e5d9-4a86-845e-8ce808a3672c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:41.384081 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1da5b4-e5d9-4a86-845e-8ce808a3672c None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:41.476056 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:41.486331 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 632/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jun 23 09:07:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 09:07:41.492158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310763a-bdbf-4e37-90d0-8468aa91836c None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:41.552001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:41.589598 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310763a-bdbf-4e37-90d0-8468aa91836c None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:41.621979 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98135dc6-31af-4204-918e-ec97b2c88ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:41.622770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98135dc6-31af-4204-918e-ec97b2c88ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:41.623098 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98135dc6-31af-4204-918e-ec97b2c88ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:41.623384 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98135dc6-31af-4204-918e-ec97b2c88ae4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:41.633002 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98135dc6-31af-4204-918e-ec97b2c88ae4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:41.634470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98135dc6-31af-4204-918e-ec97b2c88ae4 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:41.652071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-98135dc6-31af-4204-918e-ec97b2c88ae4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:41.652071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:41.652071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:41.652071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:41.652071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:41.652071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:41.652071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:41.652071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:41.652071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:41.652071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:41.652071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:41.652071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:41.652071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 09:07:41.652071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 09:07:41.652071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 09:07:41.653247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 09:07:41.653247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:41.653247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:41.653247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:41.653247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:41.653247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:41.653247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:41.653247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:41.653247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:41.653247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:41.653247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:41.653247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:41.653247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:41.653247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:41.653247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:41.653247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:41.653247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:41.654435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 09:07:41.654435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 09:07:41.654435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 09:07:41.654435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:07:41.654435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 651/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:41.659015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1da5b4-e5d9-4a86-845e-8ce808a3672c None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:41.755509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1da5b4-e5d9-4a86-845e-8ce808a3672c None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:41.789261 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18f47b-470d-4544-ab67-aa4009520f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:41.789922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18f47b-470d-4544-ab67-aa4009520f0d None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:41.790209 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18f47b-470d-4544-ab67-aa4009520f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:41.790486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18f47b-470d-4544-ab67-aa4009520f0d None admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:41.791263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18f47b-470d-4544-ab67-aa4009520f0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:41.791573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18f47b-470d-4544-ab67-aa4009520f0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:41.792956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18f47b-470d-4544-ab67-aa4009520f0d None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:41.803996 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 633/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jun 23 09:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:41.810288 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98135dc6-31af-4204-918e-ec97b2c88ae4 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:41.907194 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98135dc6-31af-4204-918e-ec97b2c88ae4 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:41.920782 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:41.940380 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7a20e-fe47-433d-8fe0-32f1cfeee204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:41.941157 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7a20e-fe47-433d-8fe0-32f1cfeee204 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:41.941442 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7a20e-fe47-433d-8fe0-32f1cfeee204 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:41.941717 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7a20e-fe47-433d-8fe0-32f1cfeee204 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:41.948773 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7a20e-fe47-433d-8fe0-32f1cfeee204 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:41.950215 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7a20e-fe47-433d-8fe0-32f1cfeee204 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:41.964130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application [None req-fdf7a20e-fe47-433d-8fe0-32f1cfeee204 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:07:41.964130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:41.964130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:41.964130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:41.964130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:41.964130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:41.964130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:41.964130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:41.964130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:41.964130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:41.964130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:41.964130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:41.964130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 09:07:41.964130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 09:07:41.964130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 09:07:41.965249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 09:07:41.965249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:41.965249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:41.965249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:41.965249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:41.965249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:41.965249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:41.965249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:41.965249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:41.965249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:41.965249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:41.965249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:41.965249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:41.965249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:41.965249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:41.965249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:41.965249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:41.966275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 09:07:41.966275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 09:07:41.966275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 09:07:41.966275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 09:07:41.966275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 09:07:41.966275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 09:07:41.966275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 09:07:41.966275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: ERROR keystone.server.flask.application Jun 23 09:07:41.966275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 652/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jun 23 09:07:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:41.971164 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18f47b-470d-4544-ab67-aa4009520f0d None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:42.020222 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:42.067831 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18f47b-470d-4544-ab67-aa4009520f0d None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:42.099733 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855766bd-eadc-4129-82d7-8d1bfda1a928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:42.100382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855766bd-eadc-4129-82d7-8d1bfda1a928 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:42.125267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855766bd-eadc-4129-82d7-8d1bfda1a928 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:42.125267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855766bd-eadc-4129-82d7-8d1bfda1a928 None admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:42.125267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855766bd-eadc-4129-82d7-8d1bfda1a928 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:42.125267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855766bd-eadc-4129-82d7-8d1bfda1a928 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:42.125267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855766bd-eadc-4129-82d7-8d1bfda1a928 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:42.125267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 634/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jun 23 09:07:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:42.125804 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7a20e-fe47-433d-8fe0-32f1cfeee204 None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:42.223011 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7a20e-fe47-433d-8fe0-32f1cfeee204 None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:42.256688 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfaa22-3298-4d0d-a62e-6470da0114d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:42.257342 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfaa22-3298-4d0d-a62e-6470da0114d8 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:42.257629 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfaa22-3298-4d0d-a62e-6470da0114d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:42.257907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfaa22-3298-4d0d-a62e-6470da0114d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:42.258740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfaa22-3298-4d0d-a62e-6470da0114d8 None admin] RBAC: Adding query filter params (role.id=9cd080b50c6f4ee8a54bf9ef1655bd83, scope.project.id=93b5040533524222b126bbe93afc56d1, user.id=98a63b20ceca40e6bceb56eda8ae9686) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:42.259081 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfaa22-3298-4d0d-a62e-6470da0114d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:42.261083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfaa22-3298-4d0d-a62e-6470da0114d8 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:42.266885 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 653/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jun 23 09:07:42 2022] GET /identity/v3/role_assignments?user.id=98a63b20ceca40e6bceb56eda8ae9686&scope.project.id=93b5040533524222b126bbe93afc56d1&role.id=9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:44.324432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eda05b-34a0-4e59-b6cb-8c506b7e836a None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:44.324987 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eda05b-34a0-4e59-b6cb-8c506b7e836a None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:44.324987 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eda05b-34a0-4e59-b6cb-8c506b7e836a None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:44.326352 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 635/1288] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:07:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:07:44.331446 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a462a15-16da-4582-8363-ae35cfd5ee34 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:44.331891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a462a15-16da-4582-8363-ae35cfd5ee34 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:44.332024 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a462a15-16da-4582-8363-ae35cfd5ee34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:44.349336 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-1a462a15-16da-4582-8363-ae35cfd5ee34 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:07:44.351897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-1a462a15-16da-4582-8363-ae35cfd5ee34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:07:44.378857 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:44.415805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 654/1289] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:07:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:07:44.438054 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eda05b-34a0-4e59-b6cb-8c506b7e836a None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:44.542074 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eda05b-34a0-4e59-b6cb-8c506b7e836a None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:44.553554 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:44.576632 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba80262-dcc0-426e-8aad-78cbd36e6a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:44.577433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba80262-dcc0-426e-8aad-78cbd36e6a17 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:44.577722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba80262-dcc0-426e-8aad-78cbd36e6a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:44.578002 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba80262-dcc0-426e-8aad-78cbd36e6a17 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:44.578685 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba80262-dcc0-426e-8aad-78cbd36e6a17 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:44.580202 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba80262-dcc0-426e-8aad-78cbd36e6a17 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:44.592253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-eba80262-dcc0-426e-8aad-78cbd36e6a17 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 09:07:44.592253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:44.592253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:07:44.592253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:07:44.592253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:07:44.592253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:07:44.592253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:07:44.592253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:07:44.592253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 09:07:44.592253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:44.592253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:07:44.592253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:44.592253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:44.592253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:44.592253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:44.592253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:44.592253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:44.593593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:44.593593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:44.593593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:44.593593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:44.593593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:44.593593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:44.593593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:07:44.593593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:07:44.593593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:07:44.593593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:07:44.593593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:44.593593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:44.593593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:44.593593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:44.593593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:44.593593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:44.594757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:44.594757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:44.594757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:44.594757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:44.594757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:44.594757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:44.594757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:44.594757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:44.594757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:44.594757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:44.594757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:07:44.594757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:07:44.594757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 09:07:44.594757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:44.594757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 636/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jun 23 09:07:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:44.599764 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a462a15-16da-4582-8363-ae35cfd5ee34 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:44.695311 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a462a15-16da-4582-8363-ae35cfd5ee34 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:44.728098 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1dd3ff-ba2e-4c3c-b599-26248bfaa3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:44.729347 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dd3ff-ba2e-4c3c-b599-26248bfaa3af None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:44.729636 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dd3ff-ba2e-4c3c-b599-26248bfaa3af None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:44.729917 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dd3ff-ba2e-4c3c-b599-26248bfaa3af None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:44.730627 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1dd3ff-ba2e-4c3c-b599-26248bfaa3af None admin] RBAC: Adding query filter params (name=placement) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:44.730956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1dd3ff-ba2e-4c3c-b599-26248bfaa3af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:44.732325 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1dd3ff-ba2e-4c3c-b599-26248bfaa3af None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:44.737119 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 655/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Jun 23 09:07:44 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:44.743191 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba80262-dcc0-426e-8aad-78cbd36e6a17 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:44.839043 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba80262-dcc0-426e-8aad-78cbd36e6a17 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:44.871448 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef442b3e-0d4f-445f-b1a9-2ac49d03a326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:44.872107 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef442b3e-0d4f-445f-b1a9-2ac49d03a326 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:44.872400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef442b3e-0d4f-445f-b1a9-2ac49d03a326 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:44.872707 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef442b3e-0d4f-445f-b1a9-2ac49d03a326 None admin] PATH_INFO: `/v3/services` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:44.873404 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef442b3e-0d4f-445f-b1a9-2ac49d03a326 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:44.873710 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef442b3e-0d4f-445f-b1a9-2ac49d03a326 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:44.875105 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef442b3e-0d4f-445f-b1a9-2ac49d03a326 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:44.879426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 637/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Jun 23 09:07:44 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:46.897537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4181080-a5bd-40bb-9691-ee7db5ec0933 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:46.898111 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4181080-a5bd-40bb-9691-ee7db5ec0933 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:46.898111 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4181080-a5bd-40bb-9691-ee7db5ec0933 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:46.899486 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 656/1293] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:07:46.904571 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa9049-e8d5-4e9e-b71d-f44491820a27 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:46.905013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa9049-e8d5-4e9e-b71d-f44491820a27 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:46.905123 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa9049-e8d5-4e9e-b71d-f44491820a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:46.922847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-6bfa9049-e8d5-4e9e-b71d-f44491820a27 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:07:46.926089 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-6bfa9049-e8d5-4e9e-b71d-f44491820a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:07:46.955035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:46.990339 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 638/1294] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:07:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:07:47.014346 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4181080-a5bd-40bb-9691-ee7db5ec0933 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:47.044528 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:47.111347 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4181080-a5bd-40bb-9691-ee7db5ec0933 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:47.143461 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66e694-4d34-433f-bf00-e14746e8cb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:47.144192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66e694-4d34-433f-bf00-e14746e8cb07 None admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:47.144480 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66e694-4d34-433f-bf00-e14746e8cb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:47.144757 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66e694-4d34-433f-bf00-e14746e8cb07 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:47.145481 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc66e694-4d34-433f-bf00-e14746e8cb07 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:47.146941 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc66e694-4d34-433f-bf00-e14746e8cb07 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:47.264558 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 657/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jun 23 09:07:47 2022] POST /identity/v3/services => generated 246 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:07:49.295972 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85710d5e-d329-4121-958a-521ee9342819 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:49.296524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85710d5e-d329-4121-958a-521ee9342819 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:49.296524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85710d5e-d329-4121-958a-521ee9342819 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:49.298143 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 639/1296] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:07:49.303348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d496106-426b-45dd-88c2-f59774ec7af6 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:49.303752 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d496106-426b-45dd-88c2-f59774ec7af6 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:49.303910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d496106-426b-45dd-88c2-f59774ec7af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:49.320796 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-3d496106-426b-45dd-88c2-f59774ec7af6 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:07:49.322006 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-3d496106-426b-45dd-88c2-f59774ec7af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:07:49.392953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 658/1297] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:07:49 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:07:49.415815 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85710d5e-d329-4121-958a-521ee9342819 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:49.523163 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85710d5e-d329-4121-958a-521ee9342819 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:49.555895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463af8b-97b2-4a47-b5f4-503e74cbb920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:49.556722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463af8b-97b2-4a47-b5f4-503e74cbb920 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:49.557523 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463af8b-97b2-4a47-b5f4-503e74cbb920 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:49.557795 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463af8b-97b2-4a47-b5f4-503e74cbb920 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:49.558437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463af8b-97b2-4a47-b5f4-503e74cbb920 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:49.559834 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463af8b-97b2-4a47-b5f4-503e74cbb920 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:49.571606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-2463af8b-97b2-4a47-b5f4-503e74cbb920 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 09:07:49.571606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:49.571606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:07:49.571606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:07:49.571606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:07:49.571606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:07:49.571606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:07:49.571606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:07:49.571606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 09:07:49.571606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:49.571606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:07:49.571606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:49.571606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:49.571606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:49.571606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:49.571606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:49.571606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:49.572902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:49.572902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:49.572902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:49.572902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:49.572902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:49.572902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:49.572902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:07:49.572902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:07:49.572902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:07:49.572902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:07:49.572902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:49.572902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:49.572902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:49.572902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:49.572902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:49.572902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:49.573913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:49.573913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:49.573913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:49.573913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:49.573913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:49.573913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:49.573913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:49.573913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:49.573913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:49.573913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:49.573913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:07:49.573913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:07:49.573913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 09:07:49.573913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:49.573913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 640/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jun 23 09:07:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:49.578877 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d496106-426b-45dd-88c2-f59774ec7af6 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:49.596974 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:49.676917 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d496106-426b-45dd-88c2-f59774ec7af6 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:49.712029 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8254be6-4ead-41ff-ae5e-32c64a13b1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:49.712693 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8254be6-4ead-41ff-ae5e-32c64a13b1ae None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:49.713008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8254be6-4ead-41ff-ae5e-32c64a13b1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:49.713287 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8254be6-4ead-41ff-ae5e-32c64a13b1ae None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:49.714249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8254be6-4ead-41ff-ae5e-32c64a13b1ae None admin] RBAC: Adding query filter params (name=placement) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:49.714558 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8254be6-4ead-41ff-ae5e-32c64a13b1ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:49.715955 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8254be6-4ead-41ff-ae5e-32c64a13b1ae None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:49.721005 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 659/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Jun 23 09:07:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:49.727151 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463af8b-97b2-4a47-b5f4-503e74cbb920 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:49.758768 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:49.823766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463af8b-97b2-4a47-b5f4-503e74cbb920 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:49.856450 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616010a-669b-4d14-bd5a-b53c0efce915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:49.857106 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616010a-669b-4d14-bd5a-b53c0efce915 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:49.857393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616010a-669b-4d14-bd5a-b53c0efce915 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:49.857670 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616010a-669b-4d14-bd5a-b53c0efce915 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:49.858462 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616010a-669b-4d14-bd5a-b53c0efce915 None admin] RBAC: Adding query filter params (interface=public, service_id=6a0d44adf5f44f4693d798cf194fd201, region_id=RegionOne) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:49.858776 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616010a-669b-4d14-bd5a-b53c0efce915 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:49.860193 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616010a-669b-4d14-bd5a-b53c0efce915 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:49.865192 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 641/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jun 23 09:07:49 2022] GET /identity/v3/endpoints?service_id=6a0d44adf5f44f4693d798cf194fd201&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:49.871349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8254be6-4ead-41ff-ae5e-32c64a13b1ae None admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:49.951310 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:49.969712 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8254be6-4ead-41ff-ae5e-32c64a13b1ae None admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:50.007119 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c09ac-fbb1-49d3-8a7a-bf92ed6955f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:50.007789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c09ac-fbb1-49d3-8a7a-bf92ed6955f7 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:50.008079 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c09ac-fbb1-49d3-8a7a-bf92ed6955f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:50.008359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c09ac-fbb1-49d3-8a7a-bf92ed6955f7 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:50.009051 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c09ac-fbb1-49d3-8a7a-bf92ed6955f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:50.010421 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c09ac-fbb1-49d3-8a7a-bf92ed6955f7 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:50.017128 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 660/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jun 23 09:07:49 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:07:52.062975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcce26-aa0b-453c-8a7c-fead915edc36 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:52.063508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcce26-aa0b-453c-8a7c-fead915edc36 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:52.063508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcce26-aa0b-453c-8a7c-fead915edc36 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:52.064946 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 642/1302] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:07:52.070096 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58164ae6-5e54-46a8-b571-d9b0f34cdb9e None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:52.070527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58164ae6-5e54-46a8-b571-d9b0f34cdb9e None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:52.070660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58164ae6-5e54-46a8-b571-d9b0f34cdb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:52.087195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-58164ae6-5e54-46a8-b571-d9b0f34cdb9e None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:07:52.088400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-58164ae6-5e54-46a8-b571-d9b0f34cdb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:07:52.156520 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 661/1303] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:07:52 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 09:07:52.179667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dcce26-aa0b-453c-8a7c-fead915edc36 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:52.274007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:52.280569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dcce26-aa0b-453c-8a7c-fead915edc36 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:52.319798 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372a814-5779-4b75-8d67-c6afa3f7d616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:52.320519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372a814-5779-4b75-8d67-c6afa3f7d616 None admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:52.320806 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372a814-5779-4b75-8d67-c6afa3f7d616 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:52.321081 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372a814-5779-4b75-8d67-c6afa3f7d616 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:52.321733 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372a814-5779-4b75-8d67-c6afa3f7d616 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:52.323707 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372a814-5779-4b75-8d67-c6afa3f7d616 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:52.335320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application [None req-1372a814-5779-4b75-8d67-c6afa3f7d616 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 09:07:52.335320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:52.335320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 09:07:52.335320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 09:07:52.335320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 09:07:52.335320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 09:07:52.335320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 09:07:52.335320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:07:52.335320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 09:07:52.335320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:52.335320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 09:07:52.335320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:52.335320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 09:07:52.335320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 09:07:52.335320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 09:07:52.335320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 09:07:52.335320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 09:07:52.336611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 09:07:52.336611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 09:07:52.336611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 09:07:52.336611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 09:07:52.336611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 09:07:52.336611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 09:07:52.336611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 09:07:52.336611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 09:07:52.336611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 09:07:52.336611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 09:07:52.336611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 09:07:52.336611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 09:07:52.336611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 09:07:52.336611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 09:07:52.336611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 09:07:52.336611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 09:07:52.337711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 09:07:52.337711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application with Lock( Jun 23 09:07:52.337711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 09:07:52.337711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self._enter() Jun 23 09:07:52.337711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 09:07:52.337711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 09:07:52.337711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 09:07:52.337711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application return self.creator() Jun 23 09:07:52.337711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 09:07:52.337711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application created_value = creator( Jun 23 09:07:52.337711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 09:07:52.337711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 09:07:52.337711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 09:07:52.337711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: ERROR keystone.server.flask.application Jun 23 09:07:52.337711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 643/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jun 23 09:07:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 09:07:52.342871 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58164ae6-5e54-46a8-b571-d9b0f34cdb9e None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:52.416958 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:07:52.438743 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58164ae6-5e54-46a8-b571-d9b0f34cdb9e None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:52.471218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364c462-3317-436b-932e-eea414682c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:52.471873 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364c462-3317-436b-932e-eea414682c50 None admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:52.472163 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364c462-3317-436b-932e-eea414682c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:52.472439 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364c462-3317-436b-932e-eea414682c50 None admin] PATH_INFO: `/v3/services` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:52.473139 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c364c462-3317-436b-932e-eea414682c50 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:07:52.473445 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c364c462-3317-436b-932e-eea414682c50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:52.474799 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c364c462-3317-436b-932e-eea414682c50 None admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:52.480539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 662/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Jun 23 09:07:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:07:52.486326 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372a814-5779-4b75-8d67-c6afa3f7d616 None admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:07:52.584287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372a814-5779-4b75-8d67-c6afa3f7d616 None admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:07:52.617868 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a0fe9-0619-453c-958b-ba4e4e37c557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:07:52.621751 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a0fe9-0619-453c-958b-ba4e4e37c557 None admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:07:52.621751 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a0fe9-0619-453c-958b-ba4e4e37c557 None admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:07:52.621751 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a0fe9-0619-453c-958b-ba4e4e37c557 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:07:52.621751 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a0fe9-0619-453c-958b-ba4e4e37c557 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:07:52.621751 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a0fe9-0619-453c-958b-ba4e4e37c557 None admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:07:52.737148 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 644/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jun 23 09:07:52 2022] POST /identity/v3/endpoints => generated 331 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:08:02.650645 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=30 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:08:02.732098 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:15:43.024310 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de877cf-2146-4e7d-8369-85cc6b996dc4 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:43.024902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de877cf-2146-4e7d-8369-85cc6b996dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:43.024902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de877cf-2146-4e7d-8369-85cc6b996dc4 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:43.031681 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45036f-974e-4457-a51a-00b64b533105 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:43.032145 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45036f-974e-4457-a51a-00b64b533105 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:43.032235 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45036f-974e-4457-a51a-00b64b533105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:43.036039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 663/1307] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:15:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:15:43.111564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-fb45036f-974e-4457-a51a-00b64b533105 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:15:43.112786 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-fb45036f-974e-4457-a51a-00b64b533105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:15:43.247334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 645/1308] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:15:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:15:43.290082 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed7e94-8b2e-4231-9090-52ec7b45ccc2 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:43.290435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed7e94-8b2e-4231-9090-52ec7b45ccc2 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:43.290649 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed7e94-8b2e-4231-9090-52ec7b45ccc2 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:43.292502 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 664/1309] 10.222.0.52 () {54 vars in 964 bytes} [Thu Jun 23 09:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:15:43.298564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4cad3-8e6a-486e-b68f-ef3974f9c3cd None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:43.298857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4cad3-8e6a-486e-b68f-ef3974f9c3cd None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:43.299157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4cad3-8e6a-486e-b68f-ef3974f9c3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:43.381314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-48f4cad3-8e6a-486e-b68f-ef3974f9c3cd None None] MFA Rules not processed for user `daed809f511a43ed8d3f88eba10466b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:15:43.382565 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-48f4cad3-8e6a-486e-b68f-ef3974f9c3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:15:43.407976 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:15:43.485381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed7e94-8b2e-4231-9090-52ec7b45ccc2 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:15:43.487065 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-87ed7e94-8b2e-4231-9090-52ec7b45ccc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:15:43.495886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 646/1310] 10.222.0.52 () {58 vars in 1061 bytes} [Thu Jun 23 09:15:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:15:43.605584 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed7e94-8b2e-4231-9090-52ec7b45ccc2 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:15:43.688714 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655682b-63e6-4988-83d3-899edf51c2c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:15:43.689373 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655682b-63e6-4988-83d3-899edf51c2c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:43.689658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655682b-63e6-4988-83d3-899edf51c2c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:43.689935 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655682b-63e6-4988-83d3-899edf51c2c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:43.691786 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-0655682b-63e6-4988-83d3-899edf51c2c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:15:43.748344 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:15:43.804484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655682b-63e6-4988-83d3-899edf51c2c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:15:43.805935 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655682b-63e6-4988-83d3-899edf51c2c8 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:15:43.839339 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 665/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:15:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:15:46.198835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c7203-bd8b-47bf-9f20-461ba6fe4dd9 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:46.199413 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c7203-bd8b-47bf-9f20-461ba6fe4dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:46.199512 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c7203-bd8b-47bf-9f20-461ba6fe4dd9 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:46.200825 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 647/1312] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:15:46.206691 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72440d-73f8-4b83-a130-f32b9f23d6f0 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:46.207213 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72440d-73f8-4b83-a130-f32b9f23d6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:46.207314 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72440d-73f8-4b83-a130-f32b9f23d6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:46.288179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-9e72440d-73f8-4b83-a130-f32b9f23d6f0 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:15:46.289405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-9e72440d-73f8-4b83-a130-f32b9f23d6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:15:46.385742 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 666/1313] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:15:46.424091 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c7203-bd8b-47bf-9f20-461ba6fe4dd9 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:15:46.426200 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-984c7203-bd8b-47bf-9f20-461ba6fe4dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:15:46.529926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c7203-bd8b-47bf-9f20-461ba6fe4dd9 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:15:46.619655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:15:46.622330 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157653d5-db64-4ef9-81ab-71b04938ace1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:15:46.623027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157653d5-db64-4ef9-81ab-71b04938ace1 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:46.623317 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157653d5-db64-4ef9-81ab-71b04938ace1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:46.623597 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157653d5-db64-4ef9-81ab-71b04938ace1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:46.625304 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-157653d5-db64-4ef9-81ab-71b04938ace1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:15:46.738503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157653d5-db64-4ef9-81ab-71b04938ace1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:15:46.739980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157653d5-db64-4ef9-81ab-71b04938ace1 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:15:46.758157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 648/1314] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:15:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:15:49.272731 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d45a0-bf45-4b7b-b2a8-81376a0eaadf None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:49.273278 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d45a0-bf45-4b7b-b2a8-81376a0eaadf None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:49.273278 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d45a0-bf45-4b7b-b2a8-81376a0eaadf None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:49.274577 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 667/1315] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:15:49.279803 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942ed80-a09f-47fa-97db-9a91d495ce37 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:49.280232 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942ed80-a09f-47fa-97db-9a91d495ce37 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:49.280351 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942ed80-a09f-47fa-97db-9a91d495ce37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:49.342894 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-a942ed80-a09f-47fa-97db-9a91d495ce37 None None] MFA Rules not processed for user `ee9d005c534c4b0da3bb98afdb8596a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:15:49.344117 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a942ed80-a09f-47fa-97db-9a91d495ce37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:15:49.395016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:15:49.440018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 649/1316] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:15:49 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:15:49.480812 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d45a0-bf45-4b7b-b2a8-81376a0eaadf None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:15:49.482351 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-850d45a0-bf45-4b7b-b2a8-81376a0eaadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:15:49.491978 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:15:49.583171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d45a0-bf45-4b7b-b2a8-81376a0eaadf None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:15:49.654332 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506887d3-d044-456e-a6a5-19a420e39fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:15:49.655005 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506887d3-d044-456e-a6a5-19a420e39fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:49.655295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506887d3-d044-456e-a6a5-19a420e39fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:49.655573 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506887d3-d044-456e-a6a5-19a420e39fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:49.657205 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-506887d3-d044-456e-a6a5-19a420e39fe4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:15:49.717874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:15:49.766446 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506887d3-d044-456e-a6a5-19a420e39fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:15:49.767914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506887d3-d044-456e-a6a5-19a420e39fe4 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:15:49.786456 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 668/1317] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:15:49 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:15:52.739869 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2899e-d9a8-44f3-b22c-88760c8e2c81 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:52.740469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2899e-d9a8-44f3-b22c-88760c8e2c81 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:52.740469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2899e-d9a8-44f3-b22c-88760c8e2c81 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:52.741781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 650/1318] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:15:52.747051 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd9623-788e-4c50-880f-4d4e26cca509 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:52.747484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd9623-788e-4c50-880f-4d4e26cca509 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:52.747626 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd9623-788e-4c50-880f-4d4e26cca509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:52.811970 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-bffd9623-788e-4c50-880f-4d4e26cca509 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:15:52.813190 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-bffd9623-788e-4c50-880f-4d4e26cca509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:15:52.906586 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 669/1319] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:15:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:15:52.945534 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2899e-d9a8-44f3-b22c-88760c8e2c81 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:15:52.947172 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-4ac2899e-d9a8-44f3-b22c-88760c8e2c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:15:53.047458 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2899e-d9a8-44f3-b22c-88760c8e2c81 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:15:53.118965 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768b1bc-f1e1-4e77-8cdc-cbdf1a1f335e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:15:53.119612 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768b1bc-f1e1-4e77-8cdc-cbdf1a1f335e service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:53.119895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768b1bc-f1e1-4e77-8cdc-cbdf1a1f335e service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:53.120168 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768b1bc-f1e1-4e77-8cdc-cbdf1a1f335e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:53.121776 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-9768b1bc-f1e1-4e77-8cdc-cbdf1a1f335e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:15:53.129547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:15:53.228578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9768b1bc-f1e1-4e77-8cdc-cbdf1a1f335e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:15:53.230015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9768b1bc-f1e1-4e77-8cdc-cbdf1a1f335e service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:15:53.247884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 651/1320] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:15:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:15:56.281615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba74682-2983-429b-bd3f-1d4a8989ba63 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:56.282146 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba74682-2983-429b-bd3f-1d4a8989ba63 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:56.282267 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba74682-2983-429b-bd3f-1d4a8989ba63 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:56.283720 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 670/1321] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:15:56.289588 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564f3b3-d776-4f63-a670-510a58ba03a5 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:56.290014 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564f3b3-d776-4f63-a670-510a58ba03a5 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:56.290189 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564f3b3-d776-4f63-a670-510a58ba03a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:56.355615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-8564f3b3-d776-4f63-a670-510a58ba03a5 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:15:56.356816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-8564f3b3-d776-4f63-a670-510a58ba03a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:15:56.415465 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:15:56.454730 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 652/1322] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:15:56.491403 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba74682-2983-429b-bd3f-1d4a8989ba63 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:15:56.492961 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-aba74682-2983-429b-bd3f-1d4a8989ba63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:15:56.511598 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:15:56.606255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba74682-2983-429b-bd3f-1d4a8989ba63 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:15:56.683561 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9512a50-ee37-431a-bbee-89551fe68037 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:15:56.684225 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9512a50-ee37-431a-bbee-89551fe68037 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:56.684516 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9512a50-ee37-431a-bbee-89551fe68037 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:56.684797 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9512a50-ee37-431a-bbee-89551fe68037 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:56.686448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e9512a50-ee37-431a-bbee-89551fe68037 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:15:56.774862 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:15:56.824139 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9512a50-ee37-431a-bbee-89551fe68037 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:15:56.825764 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9512a50-ee37-431a-bbee-89551fe68037 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:15:56.846746 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 671/1323] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:15:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:15:58.978301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d06c71-d385-447a-a9fe-23e52da78a48 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:58.978822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d06c71-d385-447a-a9fe-23e52da78a48 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:58.978985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d06c71-d385-447a-a9fe-23e52da78a48 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:58.980283 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 653/1324] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:15:58.985481 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9612a94-5ceb-4a94-96f7-f6edd7f23999 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:58.985878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9612a94-5ceb-4a94-96f7-f6edd7f23999 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:58.986052 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9612a94-5ceb-4a94-96f7-f6edd7f23999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:59.050402 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-a9612a94-5ceb-4a94-96f7-f6edd7f23999 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:15:59.051672 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-a9612a94-5ceb-4a94-96f7-f6edd7f23999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:15:59.147492 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 672/1325] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:15:59.198394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1fbaf-b258-438f-9d6d-1420e8582646 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:59.198713 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1fbaf-b258-438f-9d6d-1420e8582646 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:59.199033 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1fbaf-b258-438f-9d6d-1420e8582646 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:59.200579 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 654/1326] 10.222.0.52 () {54 vars in 964 bytes} [Thu Jun 23 09:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:15:59.206283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e6702-fbae-447b-b44d-ae57f29e592d None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:59.206573 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e6702-fbae-447b-b44d-ae57f29e592d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:59.206852 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e6702-fbae-447b-b44d-ae57f29e592d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:59.236819 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:15:59.280695 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-027e6702-fbae-447b-b44d-ae57f29e592d None None] MFA Rules not processed for user `daed809f511a43ed8d3f88eba10466b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:15:59.281919 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-027e6702-fbae-447b-b44d-ae57f29e592d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:15:59.357572 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 673/1327] 10.222.0.52 () {58 vars in 1061 bytes} [Thu Jun 23 09:15:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:15:59.367456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1fbaf-b258-438f-9d6d-1420e8582646 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:15:59.368976 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-35c1fbaf-b258-438f-9d6d-1420e8582646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:15:59.480964 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1fbaf-b258-438f-9d6d-1420e8582646 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:15:59.553532 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350bb27-5b3c-4535-8728-15027fad2422 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:15:59.554182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350bb27-5b3c-4535-8728-15027fad2422 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:15:59.554466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350bb27-5b3c-4535-8728-15027fad2422 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:15:59.554741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350bb27-5b3c-4535-8728-15027fad2422 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:15:59.556404 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-1350bb27-5b3c-4535-8728-15027fad2422 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:15:59.572842 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:15:59.676023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350bb27-5b3c-4535-8728-15027fad2422 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:15:59.677467 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350bb27-5b3c-4535-8728-15027fad2422 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:15:59.696082 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 655/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:15:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:16:02.020612 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36caff8-b059-4b72-bb16-556cc3b79aca None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:02.021123 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36caff8-b059-4b72-bb16-556cc3b79aca None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:02.021123 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36caff8-b059-4b72-bb16-556cc3b79aca None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:02.022470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 674/1329] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:02.027910 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7736a-8ea6-4281-b77c-3c864da4fbb8 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:02.028353 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7736a-8ea6-4281-b77c-3c864da4fbb8 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:02.028460 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7736a-8ea6-4281-b77c-3c864da4fbb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:02.095257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-ddd7736a-8ea6-4281-b77c-3c864da4fbb8 None None] MFA Rules not processed for user `ee9d005c534c4b0da3bb98afdb8596a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:02.097288 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ddd7736a-8ea6-4281-b77c-3c864da4fbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:02.164944 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:16:02.197238 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 656/1330] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:16:02 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:02.239176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36caff8-b059-4b72-bb16-556cc3b79aca None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:16:02.240716 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-c36caff8-b059-4b72-bb16-556cc3b79aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:02.354122 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36caff8-b059-4b72-bb16-556cc3b79aca None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:16:02.374135 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:16:02.426536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a9047-b6ac-46a0-afa0-e4f5920e97b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:16:02.427228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a9047-b6ac-46a0-afa0-e4f5920e97b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:02.427519 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a9047-b6ac-46a0-afa0-e4f5920e97b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:02.427802 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a9047-b6ac-46a0-afa0-e4f5920e97b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:02.429441 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-955a9047-b6ac-46a0-afa0-e4f5920e97b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:02.537610 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a9047-b6ac-46a0-afa0-e4f5920e97b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:16:02.538993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a9047-b6ac-46a0-afa0-e4f5920e97b4 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:16:02.557271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 675/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:16:02 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:16:05.290592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484188f3-5628-457b-a9ab-b58385fc1753 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:05.291220 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484188f3-5628-457b-a9ab-b58385fc1753 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:05.291220 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484188f3-5628-457b-a9ab-b58385fc1753 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:05.292639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 657/1332] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:05.297617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690c850-42db-40a4-9aed-a62e6499113d None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:05.298020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690c850-42db-40a4-9aed-a62e6499113d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:05.298193 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690c850-42db-40a4-9aed-a62e6499113d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:05.360482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:16:05.366009 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-2690c850-42db-40a4-9aed-a62e6499113d None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:05.366009 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-2690c850-42db-40a4-9aed-a62e6499113d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:05.459110 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 676/1333] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:16:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:05.495760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484188f3-5628-457b-a9ab-b58385fc1753 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:16:05.497362 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-484188f3-5628-457b-a9ab-b58385fc1753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:05.599485 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484188f3-5628-457b-a9ab-b58385fc1753 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:16:05.671032 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76faba-68d3-4c16-a47c-d6d13e0c2bf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:16:05.671677 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76faba-68d3-4c16-a47c-d6d13e0c2bf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:05.671967 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76faba-68d3-4c16-a47c-d6d13e0c2bf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:05.672254 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76faba-68d3-4c16-a47c-d6d13e0c2bf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:05.673889 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-6e76faba-68d3-4c16-a47c-d6d13e0c2bf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:05.700813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:16:05.780781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76faba-68d3-4c16-a47c-d6d13e0c2bf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:16:05.782216 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76faba-68d3-4c16-a47c-d6d13e0c2bf7 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:16:05.800162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 658/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:16:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:16:07.856584 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5b5a0-bf8d-4af9-9d4b-6052584a2e91 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:07.857155 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5b5a0-bf8d-4af9-9d4b-6052584a2e91 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:07.857155 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5b5a0-bf8d-4af9-9d4b-6052584a2e91 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:07.858506 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 677/1335] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:07.864073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd95710-2d03-4017-afc0-054089f723be None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:07.864478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd95710-2d03-4017-afc0-054089f723be None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:07.864643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd95710-2d03-4017-afc0-054089f723be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:07.929209 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-3bd95710-2d03-4017-afc0-054089f723be None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:07.930409 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-3bd95710-2d03-4017-afc0-054089f723be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:08.005012 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:16:08.032218 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 659/1336] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:16:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:08.072596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5b5a0-bf8d-4af9-9d4b-6052584a2e91 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:16:08.074118 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-6ce5b5a0-bf8d-4af9-9d4b-6052584a2e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:08.176362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5b5a0-bf8d-4af9-9d4b-6052584a2e91 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:16:08.203553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:16:08.248590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584aa0d-19be-4810-a080-bdddbbba94d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:16:08.249234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584aa0d-19be-4810-a080-bdddbbba94d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:08.249518 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584aa0d-19be-4810-a080-bdddbbba94d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:08.249793 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584aa0d-19be-4810-a080-bdddbbba94d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:08.251515 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b584aa0d-19be-4810-a080-bdddbbba94d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:08.356601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584aa0d-19be-4810-a080-bdddbbba94d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:16:08.358035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584aa0d-19be-4810-a080-bdddbbba94d8 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:16:08.376692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 678/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:16:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:16:11.053039 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201e60a-b466-42d6-bd3d-d738a4b2060c None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:11.053584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201e60a-b466-42d6-bd3d-d738a4b2060c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:11.053584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201e60a-b466-42d6-bd3d-d738a4b2060c None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:11.054971 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 660/1338] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:11.060053 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf22b43-1462-4ce5-ba9c-d4330d8f5785 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:11.060507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf22b43-1462-4ce5-ba9c-d4330d8f5785 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:11.060609 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf22b43-1462-4ce5-ba9c-d4330d8f5785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:11.124400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-0bf22b43-1462-4ce5-ba9c-d4330d8f5785 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:11.125607 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-0bf22b43-1462-4ce5-ba9c-d4330d8f5785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:11.134195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:16:11.223468 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 679/1339] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:16:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:11.259659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201e60a-b466-42d6-bd3d-d738a4b2060c None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:16:11.261174 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-1201e60a-b466-42d6-bd3d-d738a4b2060c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:11.361300 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201e60a-b466-42d6-bd3d-d738a4b2060c None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:16:11.433342 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfe5cc-d432-4b56-a6fd-bc7fe62deae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:16:11.434026 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfe5cc-d432-4b56-a6fd-bc7fe62deae5 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:11.434339 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfe5cc-d432-4b56-a6fd-bc7fe62deae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:11.434615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfe5cc-d432-4b56-a6fd-bc7fe62deae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:11.436277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-42cfe5cc-d432-4b56-a6fd-bc7fe62deae5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:11.470624 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:16:11.542726 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfe5cc-d432-4b56-a6fd-bc7fe62deae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:16:11.544123 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfe5cc-d432-4b56-a6fd-bc7fe62deae5 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:16:11.562897 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 661/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:16:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:16:15.841990 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074cadc2-70ba-49d1-87e7-7047ce3353e1 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:15.842875 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074cadc2-70ba-49d1-87e7-7047ce3353e1 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:15.843199 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074cadc2-70ba-49d1-87e7-7047ce3353e1 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:15.844613 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 680/1341] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:16:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:15.850679 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5add9-3e37-4360-a67b-ad2d2815d667 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:15.851190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5add9-3e37-4360-a67b-ad2d2815d667 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:15.854950 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5add9-3e37-4360-a67b-ad2d2815d667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:15.925383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-11e5add9-3e37-4360-a67b-ad2d2815d667 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:15.926611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-11e5add9-3e37-4360-a67b-ad2d2815d667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:16.037747 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:16:16.050939 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 662/1342] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:16.089533 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074cadc2-70ba-49d1-87e7-7047ce3353e1 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:16:16.091783 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-074cadc2-70ba-49d1-87e7-7047ce3353e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:16.186827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074cadc2-70ba-49d1-87e7-7047ce3353e1 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:16:16.191012 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64c942-6aac-48ec-a84c-b106e5e8372c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:16:16.191669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64c942-6aac-48ec-a84c-b106e5e8372c service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:16.191958 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64c942-6aac-48ec-a84c-b106e5e8372c service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:16.192240 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64c942-6aac-48ec-a84c-b106e5e8372c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:16.201176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64c942-6aac-48ec-a84c-b106e5e8372c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:16:16.202622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64c942-6aac-48ec-a84c-b106e5e8372c service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:16:16.212163 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 681/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:16:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:16:19.274035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563c46a-27c5-4617-aaca-1b5f21baedad None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:19.274579 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563c46a-27c5-4617-aaca-1b5f21baedad None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:19.274579 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563c46a-27c5-4617-aaca-1b5f21baedad None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:19.275962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 663/1344] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:19.281822 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e2f45-83dd-4862-b3c2-359d3aab035d None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:19.282351 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e2f45-83dd-4862-b3c2-359d3aab035d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:19.282439 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e2f45-83dd-4862-b3c2-359d3aab035d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:19.311748 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-f37e2f45-83dd-4862-b3c2-359d3aab035d None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:19.312988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-f37e2f45-83dd-4862-b3c2-359d3aab035d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:19.343709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:16:19.398884 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 682/1345] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:16:19 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:19.437433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563c46a-27c5-4617-aaca-1b5f21baedad None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:16:19.448345 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563c46a-27c5-4617-aaca-1b5f21baedad None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:16:19.452010 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece97669-5764-4d69-9ce7-002872c00c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:16:19.452656 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece97669-5764-4d69-9ce7-002872c00c88 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:19.452943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece97669-5764-4d69-9ce7-002872c00c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:19.453223 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece97669-5764-4d69-9ce7-002872c00c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:19.462736 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece97669-5764-4d69-9ce7-002872c00c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:16:19.464268 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece97669-5764-4d69-9ce7-002872c00c88 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:16:19.474475 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 664/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:16:19 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:16:23.811689 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7e848-e562-4d5d-9440-943ab01e3825 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:23.812275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7e848-e562-4d5d-9440-943ab01e3825 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:23.812275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7e848-e562-4d5d-9440-943ab01e3825 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:23.813616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 683/1347] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:23.818803 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d46ca-e7ee-43a3-a4f7-7b6e510a47b1 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:23.819291 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d46ca-e7ee-43a3-a4f7-7b6e510a47b1 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:23.819399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d46ca-e7ee-43a3-a4f7-7b6e510a47b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:23.838257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-cf6d46ca-e7ee-43a3-a4f7-7b6e510a47b1 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:23.839574 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-cf6d46ca-e7ee-43a3-a4f7-7b6e510a47b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:23.847942 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 665/1348] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:23.884664 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7e848-e562-4d5d-9440-943ab01e3825 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:16:23.895929 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7e848-e562-4d5d-9440-943ab01e3825 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:16:23.899662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530de085-7fce-4b7c-b531-a1e59c546517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:16:23.900314 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530de085-7fce-4b7c-b531-a1e59c546517 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:23.900605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530de085-7fce-4b7c-b531-a1e59c546517 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:23.900917 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530de085-7fce-4b7c-b531-a1e59c546517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:23.911134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530de085-7fce-4b7c-b531-a1e59c546517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:16:23.912576 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530de085-7fce-4b7c-b531-a1e59c546517 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:16:23.922495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 684/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:16:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:16:28.938974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85302619-611b-4857-ac5c-d62c0a2dc14e None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:28.939443 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85302619-611b-4857-ac5c-d62c0a2dc14e None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:28.939583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85302619-611b-4857-ac5c-d62c0a2dc14e None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:28.942731 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 666/1350] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jun 23 09:16:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:28.948517 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54453f91-7b5d-46e2-9be2-aa009f46506d None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:28.948873 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54453f91-7b5d-46e2-9be2-aa009f46506d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:28.948996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54453f91-7b5d-46e2-9be2-aa009f46506d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:29.093963 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-54453f91-7b5d-46e2-9be2-aa009f46506d None None] MFA Rules not processed for user `98a63b20ceca40e6bceb56eda8ae9686`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:29.095476 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-54453f91-7b5d-46e2-9be2-aa009f46506d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:29.175639 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 685/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jun 23 09:16:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:33.436636 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485a49f-6bc9-4ecd-8878-21131c943afc None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:33.437145 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485a49f-6bc9-4ecd-8878-21131c943afc None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:33.437145 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485a49f-6bc9-4ecd-8878-21131c943afc None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:33.438689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 667/1352] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jun 23 09:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:33.444180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d8eea-8c85-4459-84a2-08730c7be90a None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:33.444617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d8eea-8c85-4459-84a2-08730c7be90a None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:33.444838 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d8eea-8c85-4459-84a2-08730c7be90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:33.464165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-818d8eea-8c85-4459-84a2-08730c7be90a None None] MFA Rules not processed for user `98a63b20ceca40e6bceb56eda8ae9686`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:33.465377 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-818d8eea-8c85-4459-84a2-08730c7be90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:33.474544 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 686/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jun 23 09:16:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:33.832084 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=13 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:16:34.948103 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82dd2c2-b21c-4064-be75-9a8db032e20f None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:34.948742 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82dd2c2-b21c-4064-be75-9a8db032e20f None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:34.948742 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82dd2c2-b21c-4064-be75-9a8db032e20f None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:34.950084 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 668/1354] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jun 23 09:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:34.955146 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5302315-a9bf-4753-98aa-ef9bd2b6320b None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:34.955772 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5302315-a9bf-4753-98aa-ef9bd2b6320b None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:34.955772 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5302315-a9bf-4753-98aa-ef9bd2b6320b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:34.974641 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-f5302315-a9bf-4753-98aa-ef9bd2b6320b None None] MFA Rules not processed for user `98a63b20ceca40e6bceb56eda8ae9686`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:34.975886 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-f5302315-a9bf-4753-98aa-ef9bd2b6320b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:34.984702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 687/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jun 23 09:16:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:38.722091 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc68b27-2976-4dff-b8b9-9164d199c5bc None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:38.723002 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc68b27-2976-4dff-b8b9-9164d199c5bc None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:38.723445 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc68b27-2976-4dff-b8b9-9164d199c5bc None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:38.725268 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 669/1356] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:16:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:38.731346 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32924d07-333e-4325-a3c4-2516fc6c42e3 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:38.731346 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32924d07-333e-4325-a3c4-2516fc6c42e3 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:38.732090 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32924d07-333e-4325-a3c4-2516fc6c42e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:38.750445 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-32924d07-333e-4325-a3c4-2516fc6c42e3 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:38.751920 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-32924d07-333e-4325-a3c4-2516fc6c42e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:38.760306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 688/1357] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:38.818016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec620965-4308-400d-adb9-5817655ad536 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:38.818947 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec620965-4308-400d-adb9-5817655ad536 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:38.819756 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec620965-4308-400d-adb9-5817655ad536 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:38.822234 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 670/1358] 10.222.0.52 () {54 vars in 961 bytes} [Thu Jun 23 09:16:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:38.829805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e250641-cf85-4fb7-a902-082f0f3a5479 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:38.830857 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e250641-cf85-4fb7-a902-082f0f3a5479 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:38.836089 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e250641-cf85-4fb7-a902-082f0f3a5479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:38.879692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:16:38.895009 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-9e250641-cf85-4fb7-a902-082f0f3a5479 None None] MFA Rules not processed for user `ff025cb706d243778468ae4570374690`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:38.896269 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-9e250641-cf85-4fb7-a902-082f0f3a5479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:38.982091 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 689/1359] 10.222.0.52 () {58 vars in 1058 bytes} [Thu Jun 23 09:16:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:38.990806 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec620965-4308-400d-adb9-5817655ad536 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:16:39.006289 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec620965-4308-400d-adb9-5817655ad536 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:16:39.012060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50120bdf-7f10-4a73-815d-62084c4145f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:16:39.012814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50120bdf-7f10-4a73-815d-62084c4145f8 service nova] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:39.013108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50120bdf-7f10-4a73-815d-62084c4145f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:39.013386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50120bdf-7f10-4a73-815d-62084c4145f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:39.024092 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50120bdf-7f10-4a73-815d-62084c4145f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:16:39.025719 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50120bdf-7f10-4a73-815d-62084c4145f8 service nova] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:16:39.036364 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 671/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:16:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:16:41.289663 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af233802-ea9b-45d3-959d-86be514430d5 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:41.290184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af233802-ea9b-45d3-959d-86be514430d5 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:41.290271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af233802-ea9b-45d3-959d-86be514430d5 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:41.291620 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 690/1361] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:16:41.296834 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb2dc3-07d0-4d57-bbb3-f5d7413d459f None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:41.297627 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb2dc3-07d0-4d57-bbb3-f5d7413d459f None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:41.298069 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb2dc3-07d0-4d57-bbb3-f5d7413d459f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:41.316479 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-dccb2dc3-07d0-4d57-bbb3-f5d7413d459f None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:41.317676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-dccb2dc3-07d0-4d57-bbb3-f5d7413d459f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:41.325461 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 672/1362] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:16:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:41.370275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af233802-ea9b-45d3-959d-86be514430d5 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:16:41.384691 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af233802-ea9b-45d3-959d-86be514430d5 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:16:41.390159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a11031-6ab1-40d2-91eb-6cebadd7e391 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:16:41.390799 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a11031-6ab1-40d2-91eb-6cebadd7e391 service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:41.391107 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a11031-6ab1-40d2-91eb-6cebadd7e391 service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:41.391386 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a11031-6ab1-40d2-91eb-6cebadd7e391 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:41.399936 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a11031-6ab1-40d2-91eb-6cebadd7e391 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:16:41.401535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a11031-6ab1-40d2-91eb-6cebadd7e391 service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:16:41.411585 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 691/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:16:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:16:44.032817 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d5c27-6224-47f6-9cb7-12b62b3e68d0 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:44.033380 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d5c27-6224-47f6-9cb7-12b62b3e68d0 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:44.033380 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d5c27-6224-47f6-9cb7-12b62b3e68d0 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:44.034777 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 673/1364] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:44.040113 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d801107-eca7-46e9-b8be-57040c861c59 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:44.040510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d801107-eca7-46e9-b8be-57040c861c59 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:44.040691 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d801107-eca7-46e9-b8be-57040c861c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:44.058623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-5d801107-eca7-46e9-b8be-57040c861c59 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:44.059846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-5d801107-eca7-46e9-b8be-57040c861c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:44.068322 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 692/1365] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:44.118309 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16726fc8-1b0f-4a93-ad5f-b67159fd8ef4 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:44.118599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16726fc8-1b0f-4a93-ad5f-b67159fd8ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:44.118879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16726fc8-1b0f-4a93-ad5f-b67159fd8ef4 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:44.120320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 674/1366] 10.222.0.52 () {54 vars in 961 bytes} [Thu Jun 23 09:16:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:44.125676 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9d7ad-b6de-4c1e-a79c-c5229cbf8756 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:44.125975 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9d7ad-b6de-4c1e-a79c-c5229cbf8756 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:44.126263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9d7ad-b6de-4c1e-a79c-c5229cbf8756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:44.145312 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-b7a9d7ad-b6de-4c1e-a79c-c5229cbf8756 None None] MFA Rules not processed for user `ff025cb706d243778468ae4570374690`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:44.146518 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b7a9d7ad-b6de-4c1e-a79c-c5229cbf8756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:44.157442 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 693/1367] 10.222.0.52 () {58 vars in 1058 bytes} [Thu Jun 23 09:16:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:44.165549 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16726fc8-1b0f-4a93-ad5f-b67159fd8ef4 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:16:44.180299 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16726fc8-1b0f-4a93-ad5f-b67159fd8ef4 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:16:44.185707 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a445621-6bb2-4755-b70c-13002f612036 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:16:44.186350 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a445621-6bb2-4755-b70c-13002f612036 service nova] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:44.186638 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a445621-6bb2-4755-b70c-13002f612036 service nova] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:44.186993 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a445621-6bb2-4755-b70c-13002f612036 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:44.195614 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a445621-6bb2-4755-b70c-13002f612036 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:16:44.197202 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a445621-6bb2-4755-b70c-13002f612036 service nova] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:16:44.208498 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 675/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:16:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:16:46.468777 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acd0c9-8574-4b6a-926c-a4bd3656015e None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:46.469350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acd0c9-8574-4b6a-926c-a4bd3656015e None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:46.469350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acd0c9-8574-4b6a-926c-a4bd3656015e None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:46.470762 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 694/1369] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:46.476830 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69420ea2-3f6a-40fb-bc1c-5ea4c5f45211 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:46.477318 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69420ea2-3f6a-40fb-bc1c-5ea4c5f45211 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:46.477435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69420ea2-3f6a-40fb-bc1c-5ea4c5f45211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:46.495336 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-69420ea2-3f6a-40fb-bc1c-5ea4c5f45211 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:46.497326 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-69420ea2-3f6a-40fb-bc1c-5ea4c5f45211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:46.506093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 676/1370] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:16:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:46.550373 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acd0c9-8574-4b6a-926c-a4bd3656015e None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:16:46.565907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acd0c9-8574-4b6a-926c-a4bd3656015e None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:16:46.571563 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be8b3c-4964-4440-a547-fba0661b42b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:16:46.572398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be8b3c-4964-4440-a547-fba0661b42b1 service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:46.572756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be8b3c-4964-4440-a547-fba0661b42b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:46.573110 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be8b3c-4964-4440-a547-fba0661b42b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:46.582581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be8b3c-4964-4440-a547-fba0661b42b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:16:46.584286 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be8b3c-4964-4440-a547-fba0661b42b1 service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:16:46.594464 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 695/1371] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:16:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:16:46.922987 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fff23-4231-447f-aee9-d3f3ece3d186 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:46.923361 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fff23-4231-447f-aee9-d3f3ece3d186 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:46.923564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fff23-4231-447f-aee9-d3f3ece3d186 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:46.925698 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 677/1372] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:16:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:46.931630 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833dd20-6dea-458f-a47c-60bbf34c8442 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:46.932291 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833dd20-6dea-458f-a47c-60bbf34c8442 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:46.932786 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833dd20-6dea-458f-a47c-60bbf34c8442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:46.952613 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-3833dd20-6dea-458f-a47c-60bbf34c8442 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:46.954264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-3833dd20-6dea-458f-a47c-60bbf34c8442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:46.963868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 696/1373] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:16:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:47.329639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1633b25-c498-402c-9956-105c47e25efc None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:47.329639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1633b25-c498-402c-9956-105c47e25efc None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:47.329639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1633b25-c498-402c-9956-105c47e25efc None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:47.331922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 678/1374] 10.222.0.52 () {54 vars in 963 bytes} [Thu Jun 23 09:16:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:47.336713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053da82-f1ba-44de-98a3-d05444ae6b18 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:47.337140 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053da82-f1ba-44de-98a3-d05444ae6b18 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:47.337504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053da82-f1ba-44de-98a3-d05444ae6b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:47.387090 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-4053da82-f1ba-44de-98a3-d05444ae6b18 None None] MFA Rules not processed for user `c21a758d0e2645318c89a4fe8805df0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:47.388418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-4053da82-f1ba-44de-98a3-d05444ae6b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:47.461249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 697/1375] 10.222.0.52 () {58 vars in 1060 bytes} [Thu Jun 23 09:16:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:47.468832 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1633b25-c498-402c-9956-105c47e25efc None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:16:47.480340 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1633b25-c498-402c-9956-105c47e25efc None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:16:47.484135 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1a2e1-213f-47b8-8305-9b54d75426e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:16:47.484792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1a2e1-213f-47b8-8305-9b54d75426e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:47.485081 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1a2e1-213f-47b8-8305-9b54d75426e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:47.485365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1a2e1-213f-47b8-8305-9b54d75426e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:47.499728 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1a2e1-213f-47b8-8305-9b54d75426e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:16:47.501492 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1a2e1-213f-47b8-8305-9b54d75426e2 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:16:47.517293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 679/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:16:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:16:48.832496 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb9e33-a649-4537-a0d5-30d87d3d08a7 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:48.833081 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb9e33-a649-4537-a0d5-30d87d3d08a7 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:48.833203 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb9e33-a649-4537-a0d5-30d87d3d08a7 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:48.834568 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 698/1377] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:48.839914 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275394b5-73b7-4dd3-91cb-4f2f56082acb None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:48.840347 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275394b5-73b7-4dd3-91cb-4f2f56082acb None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:48.840449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275394b5-73b7-4dd3-91cb-4f2f56082acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:48.859011 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-275394b5-73b7-4dd3-91cb-4f2f56082acb None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:48.860233 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-275394b5-73b7-4dd3-91cb-4f2f56082acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:48.868724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 680/1378] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:16:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:48.912700 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb9e33-a649-4537-a0d5-30d87d3d08a7 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:16:48.926851 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb9e33-a649-4537-a0d5-30d87d3d08a7 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:16:48.932320 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5aac4-e388-49ce-b400-18da0b41bf4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:16:48.932965 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5aac4-e388-49ce-b400-18da0b41bf4a service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:48.933320 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5aac4-e388-49ce-b400-18da0b41bf4a service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:48.933541 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5aac4-e388-49ce-b400-18da0b41bf4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:48.942717 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5aac4-e388-49ce-b400-18da0b41bf4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:16:48.944480 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5aac4-e388-49ce-b400-18da0b41bf4a service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:16:48.948587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:16:48.955001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 699/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:16:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:16:51.225806 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab3245-6513-49b0-a86c-ef8ea38348e2 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:51.226324 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab3245-6513-49b0-a86c-ef8ea38348e2 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:51.226324 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab3245-6513-49b0-a86c-ef8ea38348e2 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:51.228589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 681/1380] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:51.234463 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f258a-4a1f-4ada-bd60-05fbf9727b99 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:51.234976 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f258a-4a1f-4ada-bd60-05fbf9727b99 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:51.235323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f258a-4a1f-4ada-bd60-05fbf9727b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:51.254725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-c05f258a-4a1f-4ada-bd60-05fbf9727b99 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:51.256228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-c05f258a-4a1f-4ada-bd60-05fbf9727b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:51.264805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 700/1381] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:16:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:51.311004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab3245-6513-49b0-a86c-ef8ea38348e2 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:16:51.326103 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab3245-6513-49b0-a86c-ef8ea38348e2 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:16:51.332424 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c47e4-246e-47c5-ab1a-36cb6872556b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:16:51.333072 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c47e4-246e-47c5-ab1a-36cb6872556b service nova] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:51.333362 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c47e4-246e-47c5-ab1a-36cb6872556b service nova] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:51.333644 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c47e4-246e-47c5-ab1a-36cb6872556b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:51.342309 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c47e4-246e-47c5-ab1a-36cb6872556b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:16:51.343938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c47e4-246e-47c5-ab1a-36cb6872556b service nova] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:16:51.355081 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 682/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:16:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:16:53.590646 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f3e99-4124-4f5b-8c0b-677ff229794c None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:53.591224 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f3e99-4124-4f5b-8c0b-677ff229794c None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:53.591224 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f3e99-4124-4f5b-8c0b-677ff229794c None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:53.592559 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 701/1383] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:53.599115 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c092d-ad8c-4339-8db8-65ba82301481 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:53.600920 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c092d-ad8c-4339-8db8-65ba82301481 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:53.600920 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c092d-ad8c-4339-8db8-65ba82301481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:53.617645 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-e62c092d-ad8c-4339-8db8-65ba82301481 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:53.618855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-e62c092d-ad8c-4339-8db8-65ba82301481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:53.629135 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 683/1384] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:53.669862 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f3e99-4124-4f5b-8c0b-677ff229794c None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:16:53.684238 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f3e99-4124-4f5b-8c0b-677ff229794c None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:16:53.689631 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0f369-d6e8-4a69-a0f0-82e647b6deaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:16:53.690262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0f369-d6e8-4a69-a0f0-82e647b6deaf service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:53.690546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0f369-d6e8-4a69-a0f0-82e647b6deaf service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:53.690837 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0f369-d6e8-4a69-a0f0-82e647b6deaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:53.699244 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0f369-d6e8-4a69-a0f0-82e647b6deaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:16:53.700841 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0f369-d6e8-4a69-a0f0-82e647b6deaf service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:16:53.710858 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 702/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:16:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:16:55.809818 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e6923-6b98-40e7-a161-ccec9616ab51 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:55.810585 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e6923-6b98-40e7-a161-ccec9616ab51 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:55.810585 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e6923-6b98-40e7-a161-ccec9616ab51 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:55.812005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 684/1386] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:55.817587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92acfa-3cec-4dc4-a1c9-36a60b7c42cf None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:55.818054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92acfa-3cec-4dc4-a1c9-36a60b7c42cf None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:55.818374 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92acfa-3cec-4dc4-a1c9-36a60b7c42cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:55.840503 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-0e92acfa-3cec-4dc4-a1c9-36a60b7c42cf None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:55.842012 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-0e92acfa-3cec-4dc4-a1c9-36a60b7c42cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:55.852003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 703/1387] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:16:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:55.896967 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e6923-6b98-40e7-a161-ccec9616ab51 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:16:55.911308 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e6923-6b98-40e7-a161-ccec9616ab51 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:16:55.916746 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe71d5-5e6d-4bba-8f87-309a09914997 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:16:55.917389 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe71d5-5e6d-4bba-8f87-309a09914997 service nova] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:55.917679 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe71d5-5e6d-4bba-8f87-309a09914997 service nova] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:55.917961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe71d5-5e6d-4bba-8f87-309a09914997 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:55.927325 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe71d5-5e6d-4bba-8f87-309a09914997 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:16:55.928996 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe71d5-5e6d-4bba-8f87-309a09914997 service nova] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:16:55.939343 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 685/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:16:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:16:58.350408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3956a1a-61b3-4f57-b8dc-0ed2b5e2f11a None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:58.351114 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3956a1a-61b3-4f57-b8dc-0ed2b5e2f11a None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:58.351114 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3956a1a-61b3-4f57-b8dc-0ed2b5e2f11a None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:58.352661 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 704/1389] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:16:58.361263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652a2d2-7665-42db-a0e6-331e8a457e3c None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:58.367540 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652a2d2-7665-42db-a0e6-331e8a457e3c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:58.367540 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652a2d2-7665-42db-a0e6-331e8a457e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:58.388362 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-6652a2d2-7665-42db-a0e6-331e8a457e3c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:16:58.389953 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-6652a2d2-7665-42db-a0e6-331e8a457e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:16:58.400160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 686/1390] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:16:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:16:58.457482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3956a1a-61b3-4f57-b8dc-0ed2b5e2f11a None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:16:58.476906 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3956a1a-61b3-4f57-b8dc-0ed2b5e2f11a None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:16:58.484139 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527d55c-ecc8-4139-b93f-ec60dfd1a0c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:16:58.484986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527d55c-ecc8-4139-b93f-ec60dfd1a0c0 service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:16:58.485362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527d55c-ecc8-4139-b93f-ec60dfd1a0c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:16:58.485738 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527d55c-ecc8-4139-b93f-ec60dfd1a0c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:16:58.497031 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a527d55c-ecc8-4139-b93f-ec60dfd1a0c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:16:58.499237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a527d55c-ecc8-4139-b93f-ec60dfd1a0c0 service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:16:58.512577 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 705/1391] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:16:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:00.852032 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aadd9b-e67a-4044-869c-cd3a8bd4b83a None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:00.852599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aadd9b-e67a-4044-869c-cd3a8bd4b83a None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:00.852599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aadd9b-e67a-4044-869c-cd3a8bd4b83a None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:00.854058 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 687/1392] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:00.859874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae74a3-a29f-4795-9593-c95cdd626e50 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:00.860476 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae74a3-a29f-4795-9593-c95cdd626e50 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:00.860476 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae74a3-a29f-4795-9593-c95cdd626e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:00.879177 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-9bae74a3-a29f-4795-9593-c95cdd626e50 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:00.880385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-9bae74a3-a29f-4795-9593-c95cdd626e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:00.888695 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 706/1393] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:17:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:00.935438 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aadd9b-e67a-4044-869c-cd3a8bd4b83a None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:00.949901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aadd9b-e67a-4044-869c-cd3a8bd4b83a None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:00.955425 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8af7f-9126-4434-a246-39c284d972fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:00.956060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8af7f-9126-4434-a246-39c284d972fc service nova] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:00.956369 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8af7f-9126-4434-a246-39c284d972fc service nova] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:00.956635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8af7f-9126-4434-a246-39c284d972fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:00.965090 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8af7f-9126-4434-a246-39c284d972fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:00.966684 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8af7f-9126-4434-a246-39c284d972fc service nova] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:00.977039 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 688/1394] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:03.160094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe053a-9a82-4f1b-af4b-3b99b1e7bc1e None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:03.160539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe053a-9a82-4f1b-af4b-3b99b1e7bc1e None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:03.160641 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe053a-9a82-4f1b-af4b-3b99b1e7bc1e None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:03.162109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 707/1395] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:17:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:03.167693 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10e863-ba52-460d-992f-d85fa72f5fad None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:03.168139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10e863-ba52-460d-992f-d85fa72f5fad None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:03.168269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10e863-ba52-460d-992f-d85fa72f5fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:03.187273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-db10e863-ba52-460d-992f-d85fa72f5fad None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:03.188465 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-db10e863-ba52-460d-992f-d85fa72f5fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:03.196233 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 689/1396] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:03.215898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc3208-b680-4ad5-9fcf-a87c194a65ee None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:03.216195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc3208-b680-4ad5-9fcf-a87c194a65ee None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:03.216479 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc3208-b680-4ad5-9fcf-a87c194a65ee None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:03.217977 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 708/1397] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:03.223961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a986d-a86f-4a2d-a9d1-2666a23b363a None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:03.224256 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a986d-a86f-4a2d-a9d1-2666a23b363a None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:03.224538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a986d-a86f-4a2d-a9d1-2666a23b363a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:03.240456 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc3208-b680-4ad5-9fcf-a87c194a65ee None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:03.243412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-5a4a986d-a86f-4a2d-a9d1-2666a23b363a None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:03.244621 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5a4a986d-a86f-4a2d-a9d1-2666a23b363a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:03.253064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 690/1398] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:03.255047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc3208-b680-4ad5-9fcf-a87c194a65ee None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:03.260504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8ac4b-9f76-4cb7-9df5-7be47c9ab8b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:03.261139 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8ac4b-9f76-4cb7-9df5-7be47c9ab8b9 service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:03.261424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8ac4b-9f76-4cb7-9df5-7be47c9ab8b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:03.261698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8ac4b-9f76-4cb7-9df5-7be47c9ab8b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:03.270376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8ac4b-9f76-4cb7-9df5-7be47c9ab8b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:03.271970 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8ac4b-9f76-4cb7-9df5-7be47c9ab8b9 service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:03.281330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 709/1399] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:03.309917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7f9c0-e2e0-44d0-bfb0-6a23ec5e0f4b None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:03.310213 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7f9c0-e2e0-44d0-bfb0-6a23ec5e0f4b None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:03.310503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7f9c0-e2e0-44d0-bfb0-6a23ec5e0f4b None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:03.311920 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 691/1400] 10.222.0.52 () {54 vars in 962 bytes} [Thu Jun 23 09:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:03.317603 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129eb4b-0f35-410b-aa6f-535c2b328243 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:03.317896 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129eb4b-0f35-410b-aa6f-535c2b328243 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:03.318179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129eb4b-0f35-410b-aa6f-535c2b328243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:03.367439 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-9129eb4b-0f35-410b-aa6f-535c2b328243 None None] MFA Rules not processed for user `36a6a3190dad4948a505bbe4c7095699`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:03.368636 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-9129eb4b-0f35-410b-aa6f-535c2b328243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:03.430102 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 710/1401] 10.222.0.52 () {58 vars in 1059 bytes} [Thu Jun 23 09:17:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:03.437638 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7f9c0-e2e0-44d0-bfb0-6a23ec5e0f4b None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:03.448910 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7f9c0-e2e0-44d0-bfb0-6a23ec5e0f4b None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:03.452792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56863dc7-6553-48c5-b6d5-0dd85a739a9f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36a6a3190dad4948a505bbe4c7095699', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:03.453438 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56863dc7-6553-48c5-b6d5-0dd85a739a9f service glance] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:03.453730 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56863dc7-6553-48c5-b6d5-0dd85a739a9f service glance] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:03.454012 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56863dc7-6553-48c5-b6d5-0dd85a739a9f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:03.463680 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56863dc7-6553-48c5-b6d5-0dd85a739a9f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:03.465163 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56863dc7-6553-48c5-b6d5-0dd85a739a9f service glance] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:03.477500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 692/1402] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:04.329805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d940dd-372a-45f4-ba61-7299d46a11b7 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:04.330256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d940dd-372a-45f4-ba61-7299d46a11b7 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:04.330419 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d940dd-372a-45f4-ba61-7299d46a11b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:04.378569 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-69d940dd-372a-45f4-ba61-7299d46a11b7 None None] MFA Rules not processed for user `cfbaba84402d4c31bafcd12133d74c03`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:04.382575 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-69d940dd-372a-45f4-ba61-7299d46a11b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:04.430408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:17:04.449243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 711/1403] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jun 23 09:17:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:04.492424 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee704f33-fcb2-49c0-8f4a-c121e0c090c9 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:04.492917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee704f33-fcb2-49c0-8f4a-c121e0c090c9 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:04.493147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee704f33-fcb2-49c0-8f4a-c121e0c090c9 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:04.494597 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 693/1404] 10.222.0.52 () {54 vars in 942 bytes} [Thu Jun 23 09:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:04.501195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bba33-d824-4d99-a8bb-24a18bfad610 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:04.501561 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bba33-d824-4d99-a8bb-24a18bfad610 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:04.501913 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bba33-d824-4d99-a8bb-24a18bfad610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:04.553604 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-930bba33-d824-4d99-a8bb-24a18bfad610 None None] MFA Rules not processed for user `5b1331b4f0924bb8b137a22a33c9d130`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:04.554857 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-930bba33-d824-4d99-a8bb-24a18bfad610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:04.626708 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 712/1405] 10.222.0.52 () {58 vars in 1039 bytes} [Thu Jun 23 09:17:04 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:04.636491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee704f33-fcb2-49c0-8f4a-c121e0c090c9 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:04.644439 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=64 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:17:04.650921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee704f33-fcb2-49c0-8f4a-c121e0c090c9 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:04.655956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bffd20-316b-4185-9d9b-6a3ea42f72c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b1331b4f0924bb8b137a22a33c9d130', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:04.659791 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bffd20-316b-4185-9d9b-6a3ea42f72c5 service swift] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:04.659791 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bffd20-316b-4185-9d9b-6a3ea42f72c5 service swift] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:04.659791 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bffd20-316b-4185-9d9b-6a3ea42f72c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:04.665312 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bffd20-316b-4185-9d9b-6a3ea42f72c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:04.667081 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bffd20-316b-4185-9d9b-6a3ea42f72c5 service swift] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:04.675311 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 694/1406] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jun 23 09:17:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 09:17:05.506338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e746c6a-2dfd-487e-aca1-8106a822a460 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:05.506886 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e746c6a-2dfd-487e-aca1-8106a822a460 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:05.506886 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e746c6a-2dfd-487e-aca1-8106a822a460 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:05.508318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 713/1407] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:17:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:05.513598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd34a4d-eac6-48d5-90e3-a2a1d8289004 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:05.514498 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd34a4d-eac6-48d5-90e3-a2a1d8289004 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:05.515051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd34a4d-eac6-48d5-90e3-a2a1d8289004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:05.533892 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-9bd34a4d-eac6-48d5-90e3-a2a1d8289004 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:05.535236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-9bd34a4d-eac6-48d5-90e3-a2a1d8289004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:05.543363 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 695/1408] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:17:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:05.589409 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e746c6a-2dfd-487e-aca1-8106a822a460 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:05.604727 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e746c6a-2dfd-487e-aca1-8106a822a460 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:05.610438 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b4743-4bf2-4545-9852-a9488e87abbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:05.611775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b4743-4bf2-4545-9852-a9488e87abbf service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:05.612150 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b4743-4bf2-4545-9852-a9488e87abbf service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:05.612491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b4743-4bf2-4545-9852-a9488e87abbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:05.624401 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b4743-4bf2-4545-9852-a9488e87abbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:05.626127 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b4743-4bf2-4545-9852-a9488e87abbf service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:05.637630 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 714/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:15.528099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=8 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:17:15.632465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=56 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:17:28.329472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c831833-db7b-466d-b43d-6ad1dd6afcbb None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:28.330051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c831833-db7b-466d-b43d-6ad1dd6afcbb None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:28.330051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c831833-db7b-466d-b43d-6ad1dd6afcbb None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:28.331455 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 696/1410] 10.222.0.52 () {54 vars in 918 bytes} [Thu Jun 23 09:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:28.336531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d0c78-fde2-463c-80be-1fd4c2a72785 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:28.336941 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d0c78-fde2-463c-80be-1fd4c2a72785 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:28.337090 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d0c78-fde2-463c-80be-1fd4c2a72785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:28.354765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-445d0c78-fde2-463c-80be-1fd4c2a72785 None None] MFA Rules not processed for user `98a63b20ceca40e6bceb56eda8ae9686`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:28.355981 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-445d0c78-fde2-463c-80be-1fd4c2a72785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:28.364936 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 715/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Thu Jun 23 09:17:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:28.379580 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0f76c-3d74-4cdc-9abe-f967cab8a873 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:28.379869 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0f76c-3d74-4cdc-9abe-f967cab8a873 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:28.380148 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0f76c-3d74-4cdc-9abe-f967cab8a873 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:28.381474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 697/1412] 10.222.0.52 () {54 vars in 918 bytes} [Thu Jun 23 09:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:28.385994 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1cf9a-49d2-4d2e-8592-17bdb9dbc556 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:28.386288 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1cf9a-49d2-4d2e-8592-17bdb9dbc556 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:28.386570 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1cf9a-49d2-4d2e-8592-17bdb9dbc556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:28.403993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-01f1cf9a-49d2-4d2e-8592-17bdb9dbc556 None None] MFA Rules not processed for user `ff025cb706d243778468ae4570374690`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:28.405227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-01f1cf9a-49d2-4d2e-8592-17bdb9dbc556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:28.414984 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 716/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Thu Jun 23 09:17:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:17:30.903491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798ea5c-6cda-4329-a7f6-bf25d86419a9 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:30.904070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798ea5c-6cda-4329-a7f6-bf25d86419a9 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:30.904070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798ea5c-6cda-4329-a7f6-bf25d86419a9 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:30.905398 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 09:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:30.911057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d74d3-5644-43c9-97f8-3c3bdaa9f293 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:30.911469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d74d3-5644-43c9-97f8-3c3bdaa9f293 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:30.911685 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d74d3-5644-43c9-97f8-3c3bdaa9f293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:30.929669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-7d5d74d3-5644-43c9-97f8-3c3bdaa9f293 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:30.930861 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7d5d74d3-5644-43c9-97f8-3c3bdaa9f293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:30.938595 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 09:17:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:30.983764 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798ea5c-6cda-4329-a7f6-bf25d86419a9 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:30.998700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798ea5c-6cda-4329-a7f6-bf25d86419a9 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:31.004618 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c7545-a967-496d-ba39-2b2a41ec2c7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:31.005260 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c7545-a967-496d-ba39-2b2a41ec2c7b service nova] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:31.005550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c7545-a967-496d-ba39-2b2a41ec2c7b service nova] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:31.005830 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c7545-a967-496d-ba39-2b2a41ec2c7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:31.014073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c7545-a967-496d-ba39-2b2a41ec2c7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:31.015718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c7545-a967-496d-ba39-2b2a41ec2c7b service nova] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:31.025709 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 699/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:31.915893 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258275b-ad0c-4e1e-8aed-7c14baddd879 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:31.916461 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258275b-ad0c-4e1e-8aed-7c14baddd879 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:31.916461 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258275b-ad0c-4e1e-8aed-7c14baddd879 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:31.917844 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 09:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:31.922739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bb436-9f0a-4852-be3f-15a038a1f9d0 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:31.923182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bb436-9f0a-4852-be3f-15a038a1f9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:31.923977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bb436-9f0a-4852-be3f-15a038a1f9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:31.942057 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-575bb436-9f0a-4852-be3f-15a038a1f9d0 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:31.943291 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-575bb436-9f0a-4852-be3f-15a038a1f9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:31.950999 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 09:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:32.101704 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258275b-ad0c-4e1e-8aed-7c14baddd879 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:32.116671 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258275b-ad0c-4e1e-8aed-7c14baddd879 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:32.122066 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4f39a-27ec-4306-9220-9f0f0538d6f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:32.122711 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4f39a-27ec-4306-9220-9f0f0538d6f3 service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:32.123037 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4f39a-27ec-4306-9220-9f0f0538d6f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:32.123321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4f39a-27ec-4306-9220-9f0f0538d6f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:32.132296 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4f39a-27ec-4306-9220-9f0f0538d6f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:32.133892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4f39a-27ec-4306-9220-9f0f0538d6f3 service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:32.143331 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 719/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:33.028027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ad1a7-07fd-4069-8805-ca0aefd9b7b2 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:33.028834 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ad1a7-07fd-4069-8805-ca0aefd9b7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:33.028834 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ad1a7-07fd-4069-8805-ca0aefd9b7b2 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:33.029952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 09:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:33.035532 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cb260-7269-4119-a549-be778c2f35b3 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:33.036050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cb260-7269-4119-a549-be778c2f35b3 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:33.036152 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cb260-7269-4119-a549-be778c2f35b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:33.058779 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-b62cb260-7269-4119-a549-be778c2f35b3 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:33.060028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b62cb260-7269-4119-a549-be778c2f35b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:33.068365 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 09:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:33.218156 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ad1a7-07fd-4069-8805-ca0aefd9b7b2 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:33.232413 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ad1a7-07fd-4069-8805-ca0aefd9b7b2 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:33.237687 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4952955-c2be-4f21-b3d7-138016453f21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:33.238318 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4952955-c2be-4f21-b3d7-138016453f21 service nova] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:33.238606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4952955-c2be-4f21-b3d7-138016453f21 service nova] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:33.238901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4952955-c2be-4f21-b3d7-138016453f21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:33.246939 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4952955-c2be-4f21-b3d7-138016453f21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:33.248482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4952955-c2be-4f21-b3d7-138016453f21 service nova] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:33.258328 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 702/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:34.115718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a51eca-ff1e-4abf-8b6f-ee11b2d6b2f3 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:34.116306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a51eca-ff1e-4abf-8b6f-ee11b2d6b2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:34.116306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a51eca-ff1e-4abf-8b6f-ee11b2d6b2f3 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:34.117665 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 09:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:34.122565 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495fa8d-7e2c-4adb-b6db-39bbff00c69a None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:34.123003 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495fa8d-7e2c-4adb-b6db-39bbff00c69a None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:34.123155 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495fa8d-7e2c-4adb-b6db-39bbff00c69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:34.141549 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-2495fa8d-7e2c-4adb-b6db-39bbff00c69a None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:34.142812 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-2495fa8d-7e2c-4adb-b6db-39bbff00c69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:34.150548 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 09:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:34.299722 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a51eca-ff1e-4abf-8b6f-ee11b2d6b2f3 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:34.314594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a51eca-ff1e-4abf-8b6f-ee11b2d6b2f3 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:34.320055 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42b544-873e-420d-a633-63324746fa7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:34.320698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42b544-873e-420d-a633-63324746fa7b service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:34.320987 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42b544-873e-420d-a633-63324746fa7b service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:34.321264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42b544-873e-420d-a633-63324746fa7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:34.329272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42b544-873e-420d-a633-63324746fa7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:34.330866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42b544-873e-420d-a633-63324746fa7b service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:34.340198 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 722/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:35.206639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88384c26-9bf0-45fc-bff7-da500aa09e0e None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:35.207182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88384c26-9bf0-45fc-bff7-da500aa09e0e None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:35.207288 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88384c26-9bf0-45fc-bff7-da500aa09e0e None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:35.208644 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 09:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:35.213584 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2e0ed-ec80-4715-9280-e472a00d73de None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:35.214029 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2e0ed-ec80-4715-9280-e472a00d73de None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:35.214134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2e0ed-ec80-4715-9280-e472a00d73de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:35.232561 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-87b2e0ed-ec80-4715-9280-e472a00d73de None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:35.233761 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-87b2e0ed-ec80-4715-9280-e472a00d73de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:35.242190 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 09:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:35.391505 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88384c26-9bf0-45fc-bff7-da500aa09e0e None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:35.405747 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88384c26-9bf0-45fc-bff7-da500aa09e0e None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:35.412029 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673fa1d-7750-4982-b69d-a8ed3dd6379f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:35.412698 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673fa1d-7750-4982-b69d-a8ed3dd6379f service nova] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:35.412988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673fa1d-7750-4982-b69d-a8ed3dd6379f service nova] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:35.413268 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673fa1d-7750-4982-b69d-a8ed3dd6379f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:35.421542 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7673fa1d-7750-4982-b69d-a8ed3dd6379f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:35.423175 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7673fa1d-7750-4982-b69d-a8ed3dd6379f service nova] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:35.433087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 705/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:36.298314 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944f8c1-f6a9-4cd4-85ae-4be245ef9e5b None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:36.298959 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944f8c1-f6a9-4cd4-85ae-4be245ef9e5b None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:36.298959 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944f8c1-f6a9-4cd4-85ae-4be245ef9e5b None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:36.300360 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 09:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:36.305248 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062c4ed-b460-41d6-82d9-a7d94b1c4183 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:36.305653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062c4ed-b460-41d6-82d9-a7d94b1c4183 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:36.305836 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062c4ed-b460-41d6-82d9-a7d94b1c4183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:36.323290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-5062c4ed-b460-41d6-82d9-a7d94b1c4183 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:36.324470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5062c4ed-b460-41d6-82d9-a7d94b1c4183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:36.332684 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 09:17:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:36.483767 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944f8c1-f6a9-4cd4-85ae-4be245ef9e5b None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:36.497821 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944f8c1-f6a9-4cd4-85ae-4be245ef9e5b None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:36.503192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6325b9e-5d03-4bea-96ba-378d6e9b0fca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:36.503828 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6325b9e-5d03-4bea-96ba-378d6e9b0fca service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:36.504112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6325b9e-5d03-4bea-96ba-378d6e9b0fca service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:36.504386 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6325b9e-5d03-4bea-96ba-378d6e9b0fca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:36.512275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6325b9e-5d03-4bea-96ba-378d6e9b0fca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:36.513851 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6325b9e-5d03-4bea-96ba-378d6e9b0fca service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:36.523486 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 725/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:37.402671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789c2bc-87a9-454f-b099-bb27517fcd34 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:37.403261 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789c2bc-87a9-454f-b099-bb27517fcd34 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:37.403690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789c2bc-87a9-454f-b099-bb27517fcd34 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:37.406113 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 09:17:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:37.412116 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290e3bc-495c-4183-9ef4-d2a36c142607 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:37.412583 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290e3bc-495c-4183-9ef4-d2a36c142607 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:37.412685 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290e3bc-495c-4183-9ef4-d2a36c142607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:37.430613 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-4290e3bc-495c-4183-9ef4-d2a36c142607 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:37.431837 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-4290e3bc-495c-4183-9ef4-d2a36c142607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:37.439874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 09:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:37.589465 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789c2bc-87a9-454f-b099-bb27517fcd34 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:37.604361 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789c2bc-87a9-454f-b099-bb27517fcd34 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:37.609623 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba797b-86bd-4aff-8bfc-416e8e5a5c23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:37.610260 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba797b-86bd-4aff-8bfc-416e8e5a5c23 service nova] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:37.610548 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba797b-86bd-4aff-8bfc-416e8e5a5c23 service nova] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:37.611389 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba797b-86bd-4aff-8bfc-416e8e5a5c23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:37.619352 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba797b-86bd-4aff-8bfc-416e8e5a5c23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:37.620953 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba797b-86bd-4aff-8bfc-416e8e5a5c23 service nova] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:37.630733 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 708/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:38.515780 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7b06b-5931-49eb-bb1c-398389fff72f None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:38.516238 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7b06b-5931-49eb-bb1c-398389fff72f None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:38.516367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7b06b-5931-49eb-bb1c-398389fff72f None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:38.517728 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 09:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:38.522656 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71364688-640e-4812-9c81-027c21977816 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:38.523118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71364688-640e-4812-9c81-027c21977816 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:38.523260 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71364688-640e-4812-9c81-027c21977816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:38.542239 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-71364688-640e-4812-9c81-027c21977816 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:38.543576 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-71364688-640e-4812-9c81-027c21977816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:38.552032 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 09:17:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:38.706184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7b06b-5931-49eb-bb1c-398389fff72f None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:38.720596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7b06b-5931-49eb-bb1c-398389fff72f None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:38.726271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad180e-e67a-4a3f-b86a-b96777796345 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:38.726922 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad180e-e67a-4a3f-b86a-b96777796345 service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:38.727225 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad180e-e67a-4a3f-b86a-b96777796345 service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:38.727504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad180e-e67a-4a3f-b86a-b96777796345 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:38.736473 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad180e-e67a-4a3f-b86a-b96777796345 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:38.740220 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad180e-e67a-4a3f-b86a-b96777796345 service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:38.751330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 728/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:39.644094 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10f6c0-5a72-4d3d-a469-6b0be0de4538 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:39.644639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10f6c0-5a72-4d3d-a469-6b0be0de4538 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:39.644743 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10f6c0-5a72-4d3d-a469-6b0be0de4538 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:39.646059 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 09:17:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:39.650806 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75426fc5-b29d-406e-97bb-95c09d2f2890 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:39.651222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75426fc5-b29d-406e-97bb-95c09d2f2890 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:39.651407 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75426fc5-b29d-406e-97bb-95c09d2f2890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:39.669327 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-75426fc5-b29d-406e-97bb-95c09d2f2890 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:39.670513 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-75426fc5-b29d-406e-97bb-95c09d2f2890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:39.678215 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 09:17:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:39.826879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a10f6c0-5a72-4d3d-a469-6b0be0de4538 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:39.840987 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a10f6c0-5a72-4d3d-a469-6b0be0de4538 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:39.846825 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ecaeea-7666-4efa-b569-6ad42d9d35e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:39.847479 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecaeea-7666-4efa-b569-6ad42d9d35e9 service nova] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:39.847762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecaeea-7666-4efa-b569-6ad42d9d35e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:39.848278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecaeea-7666-4efa-b569-6ad42d9d35e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:39.856223 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecaeea-7666-4efa-b569-6ad42d9d35e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:39.857800 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecaeea-7666-4efa-b569-6ad42d9d35e9 service nova] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:39.867556 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 711/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:41.806516 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6e3af-1768-4013-8434-454679d4b137 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:41.807140 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6e3af-1768-4013-8434-454679d4b137 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:41.807140 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6e3af-1768-4013-8434-454679d4b137 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:41.808490 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 09:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:41.813811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ad7d8-562a-4084-b19f-f3f3f8f72f56 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:41.814794 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ad7d8-562a-4084-b19f-f3f3f8f72f56 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:41.814928 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ad7d8-562a-4084-b19f-f3f3f8f72f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:41.832145 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-2f4ad7d8-562a-4084-b19f-f3f3f8f72f56 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:41.833322 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-2f4ad7d8-562a-4084-b19f-f3f3f8f72f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:41.840911 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 09:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:41.879622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6e3af-1768-4013-8434-454679d4b137 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:41.890142 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6e3af-1768-4013-8434-454679d4b137 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:41.893640 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c806a2-e7c0-4c5b-a120-70283816a327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:41.894278 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c806a2-e7c0-4c5b-a120-70283816a327 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:41.894566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c806a2-e7c0-4c5b-a120-70283816a327 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:41.894848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c806a2-e7c0-4c5b-a120-70283816a327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:41.903720 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c806a2-e7c0-4c5b-a120-70283816a327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:41.905154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c806a2-e7c0-4c5b-a120-70283816a327 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:41.914880 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 731/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:44.088179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeefa38-249d-400b-923e-6259e62e747f None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:44.088753 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeefa38-249d-400b-923e-6259e62e747f None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:44.088753 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeefa38-249d-400b-923e-6259e62e747f None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:44.090093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 09:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:44.095989 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da19332-60b3-47cf-87f0-d011a54f6335 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:44.096389 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da19332-60b3-47cf-87f0-d011a54f6335 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:44.096547 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da19332-60b3-47cf-87f0-d011a54f6335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:44.114051 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-5da19332-60b3-47cf-87f0-d011a54f6335 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:44.115274 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-5da19332-60b3-47cf-87f0-d011a54f6335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:44.122891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 09:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:44.160798 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeefa38-249d-400b-923e-6259e62e747f None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:44.171525 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeefa38-249d-400b-923e-6259e62e747f None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:44.175111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9f55f-bb8d-4377-b1ce-3afa570e0464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:44.175804 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9f55f-bb8d-4377-b1ce-3afa570e0464 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:44.176089 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9f55f-bb8d-4377-b1ce-3afa570e0464 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:44.176365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9f55f-bb8d-4377-b1ce-3afa570e0464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:44.185352 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9f55f-bb8d-4377-b1ce-3afa570e0464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:44.186785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9f55f-bb8d-4377-b1ce-3afa570e0464 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:44.196778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 714/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:46.655925 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3e1f9-9be2-42d6-afa7-4a004afee8df None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:46.656468 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3e1f9-9be2-42d6-afa7-4a004afee8df None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:46.656596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3e1f9-9be2-42d6-afa7-4a004afee8df None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:46.658404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 09:17:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:46.663798 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd24a8-4583-4595-aff5-c8ea579bdcb8 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:46.664236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd24a8-4583-4595-aff5-c8ea579bdcb8 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:46.664339 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd24a8-4583-4595-aff5-c8ea579bdcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:46.682441 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-33dd24a8-4583-4595-aff5-c8ea579bdcb8 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:46.683681 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-33dd24a8-4583-4595-aff5-c8ea579bdcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:46.691517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 09:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:46.729865 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f3e1f9-9be2-42d6-afa7-4a004afee8df None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:46.741964 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f3e1f9-9be2-42d6-afa7-4a004afee8df None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:46.745678 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ad37c-b617-452e-80a0-6680cc1d0502 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:46.746318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ad37c-b617-452e-80a0-6680cc1d0502 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:46.746602 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ad37c-b617-452e-80a0-6680cc1d0502 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:46.747088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ad37c-b617-452e-80a0-6680cc1d0502 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:46.756593 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ad37c-b617-452e-80a0-6680cc1d0502 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:46.758013 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ad37c-b617-452e-80a0-6680cc1d0502 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:46.767382 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 734/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:48.938195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57ddc8-ec29-48f2-b9d1-815abbd01894 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:48.939285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57ddc8-ec29-48f2-b9d1-815abbd01894 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:48.939285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57ddc8-ec29-48f2-b9d1-815abbd01894 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:48.940661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 09:17:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:48.946129 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4199f8-7bf4-4066-aff6-5a28b1c3f6ee None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:48.946553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4199f8-7bf4-4066-aff6-5a28b1c3f6ee None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:48.946690 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4199f8-7bf4-4066-aff6-5a28b1c3f6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:48.964878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-ca4199f8-7bf4-4066-aff6-5a28b1c3f6ee None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:48.966076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-ca4199f8-7bf4-4066-aff6-5a28b1c3f6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:48.974549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 09:17:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:49.012610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57ddc8-ec29-48f2-b9d1-815abbd01894 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:49.023115 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57ddc8-ec29-48f2-b9d1-815abbd01894 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:49.026577 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c13a8f-1a0a-4f11-adde-5c7a215bc4b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:49.027245 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c13a8f-1a0a-4f11-adde-5c7a215bc4b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:49.027527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c13a8f-1a0a-4f11-adde-5c7a215bc4b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:49.027800 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c13a8f-1a0a-4f11-adde-5c7a215bc4b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:49.036926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c13a8f-1a0a-4f11-adde-5c7a215bc4b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:49.038355 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c13a8f-1a0a-4f11-adde-5c7a215bc4b1 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:49.047911 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 717/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:51.123869 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f972bc8-ec87-4e3d-848b-3c4ac1498f84 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:51.124397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f972bc8-ec87-4e3d-848b-3c4ac1498f84 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:51.124498 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f972bc8-ec87-4e3d-848b-3c4ac1498f84 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:51.125844 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 09:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:51.131921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da19fe9-0cf5-4859-b318-d54c171af79a None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:51.132386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da19fe9-0cf5-4859-b318-d54c171af79a None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:51.132521 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da19fe9-0cf5-4859-b318-d54c171af79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:51.150458 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-1da19fe9-0cf5-4859-b318-d54c171af79a None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:51.151697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-1da19fe9-0cf5-4859-b318-d54c171af79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:51.159609 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 09:17:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:51.196973 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f972bc8-ec87-4e3d-848b-3c4ac1498f84 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:51.208125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f972bc8-ec87-4e3d-848b-3c4ac1498f84 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:51.211643 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180f234-c68c-4c6e-a00f-385d4e49cd1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:51.212283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180f234-c68c-4c6e-a00f-385d4e49cd1d service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:51.212571 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180f234-c68c-4c6e-a00f-385d4e49cd1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:51.212851 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180f234-c68c-4c6e-a00f-385d4e49cd1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:51.221569 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180f234-c68c-4c6e-a00f-385d4e49cd1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:51.223044 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180f234-c68c-4c6e-a00f-385d4e49cd1d service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:51.232520 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 737/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:53.302228 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e03c7-d3ec-451e-8b45-c15fbb3a1942 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:53.302801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e03c7-d3ec-451e-8b45-c15fbb3a1942 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:53.302801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e03c7-d3ec-451e-8b45-c15fbb3a1942 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:53.304178 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 09:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:53.309360 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3d03a-6c8e-4866-8a0b-cabb76c0762a None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:53.309758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3d03a-6c8e-4866-8a0b-cabb76c0762a None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:53.309918 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3d03a-6c8e-4866-8a0b-cabb76c0762a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:53.327855 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-aac3d03a-6c8e-4866-8a0b-cabb76c0762a None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:53.329059 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-aac3d03a-6c8e-4866-8a0b-cabb76c0762a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:53.337315 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 09:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:53.374620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e03c7-d3ec-451e-8b45-c15fbb3a1942 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:53.385619 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e03c7-d3ec-451e-8b45-c15fbb3a1942 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:53.389076 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b3bd7-f756-41c0-bedf-c8e8303053c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:53.389704 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b3bd7-f756-41c0-bedf-c8e8303053c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:53.390000 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b3bd7-f756-41c0-bedf-c8e8303053c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:53.390266 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b3bd7-f756-41c0-bedf-c8e8303053c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:53.399516 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b3bd7-f756-41c0-bedf-c8e8303053c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:53.400948 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b3bd7-f756-41c0-bedf-c8e8303053c3 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:53.410028 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 720/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:55.510387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9270481-ec75-4b57-bfd8-2b53caff6d4a None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:55.510956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9270481-ec75-4b57-bfd8-2b53caff6d4a None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:55.510956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9270481-ec75-4b57-bfd8-2b53caff6d4a None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:55.512278 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 09:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:55.517589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f940d-f39b-4138-9208-725b6f3c977c None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:55.517977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f940d-f39b-4138-9208-725b6f3c977c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:55.518180 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f940d-f39b-4138-9208-725b6f3c977c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:55.536094 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-d64f940d-f39b-4138-9208-725b6f3c977c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:55.537297 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d64f940d-f39b-4138-9208-725b6f3c977c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:55.545065 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 09:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:55.583893 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9270481-ec75-4b57-bfd8-2b53caff6d4a None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:55.594383 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9270481-ec75-4b57-bfd8-2b53caff6d4a None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:55.598356 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdabaf6-4f59-439e-9061-49b8c718c926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:55.599296 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdabaf6-4f59-439e-9061-49b8c718c926 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:55.599587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdabaf6-4f59-439e-9061-49b8c718c926 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:55.599868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdabaf6-4f59-439e-9061-49b8c718c926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:55.608529 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdabaf6-4f59-439e-9061-49b8c718c926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:55.609964 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdabaf6-4f59-439e-9061-49b8c718c926 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:55.619733 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 740/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:57.689596 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903d9a8-8930-4bdb-893c-abee1d9bde83 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:57.690128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903d9a8-8930-4bdb-893c-abee1d9bde83 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:57.690128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903d9a8-8930-4bdb-893c-abee1d9bde83 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:57.691525 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 09:17:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:57.696815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8b98d-4f51-4c8c-98ad-7321d42ebaec None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:57.697241 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8b98d-4f51-4c8c-98ad-7321d42ebaec None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:57.697420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8b98d-4f51-4c8c-98ad-7321d42ebaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:57.703463 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=79 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:17:57.715749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-c1a8b98d-4f51-4c8c-98ad-7321d42ebaec None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:57.717111 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-c1a8b98d-4f51-4c8c-98ad-7321d42ebaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:57.725774 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 09:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:57.763162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903d9a8-8930-4bdb-893c-abee1d9bde83 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:57.768780 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=73 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:17:57.774568 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903d9a8-8930-4bdb-893c-abee1d9bde83 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:57.778060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408415ba-e1ef-4cff-9939-9db24c97e81e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:57.778695 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408415ba-e1ef-4cff-9939-9db24c97e81e service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:57.779037 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408415ba-e1ef-4cff-9939-9db24c97e81e service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:57.779400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408415ba-e1ef-4cff-9939-9db24c97e81e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:57.788669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408415ba-e1ef-4cff-9939-9db24c97e81e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:57.790104 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408415ba-e1ef-4cff-9939-9db24c97e81e service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:57.799735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 723/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:17:59.883050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f707a29-af5a-45ca-8605-ce7d30ef4476 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:59.883645 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f707a29-af5a-45ca-8605-ce7d30ef4476 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:59.883645 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f707a29-af5a-45ca-8605-ce7d30ef4476 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:59.885007 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 09:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:17:59.890186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7b02b-b2a9-42de-bda7-79715f7f1bf1 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:59.890589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7b02b-b2a9-42de-bda7-79715f7f1bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:59.890780 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7b02b-b2a9-42de-bda7-79715f7f1bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:59.909268 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-bee7b02b-b2a9-42de-bda7-79715f7f1bf1 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:17:59.910462 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-bee7b02b-b2a9-42de-bda7-79715f7f1bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:17:59.918135 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 09:17:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:17:59.954935 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f707a29-af5a-45ca-8605-ce7d30ef4476 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:17:59.965585 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f707a29-af5a-45ca-8605-ce7d30ef4476 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:17:59.969133 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168102cc-c082-41a8-a6d7-67c89a4eab9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:17:59.969773 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168102cc-c082-41a8-a6d7-67c89a4eab9f service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:17:59.970091 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168102cc-c082-41a8-a6d7-67c89a4eab9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:17:59.970374 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168102cc-c082-41a8-a6d7-67c89a4eab9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:17:59.979394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168102cc-c082-41a8-a6d7-67c89a4eab9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:17:59.980834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168102cc-c082-41a8-a6d7-67c89a4eab9f service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:17:59.990165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 743/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:17:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:18:02.064184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9c7d4-dcf2-4fd7-ace9-7d117bdca911 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:02.064760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9c7d4-dcf2-4fd7-ace9-7d117bdca911 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:02.064760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9c7d4-dcf2-4fd7-ace9-7d117bdca911 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:02.066122 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 09:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:18:02.071459 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73434d-2fc8-4a63-8946-5437cedce3df None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:02.071871 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73434d-2fc8-4a63-8946-5437cedce3df None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:02.072027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73434d-2fc8-4a63-8946-5437cedce3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:02.090237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-bc73434d-2fc8-4a63-8946-5437cedce3df None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:18:02.091467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-bc73434d-2fc8-4a63-8946-5437cedce3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:18:02.099973 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 09:18:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:18:02.136342 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9c7d4-dcf2-4fd7-ace9-7d117bdca911 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:18:02.147338 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9c7d4-dcf2-4fd7-ace9-7d117bdca911 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:18:02.150870 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704c8f2-853b-4c28-979e-f721ed1f8f6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:18:02.151528 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704c8f2-853b-4c28-979e-f721ed1f8f6c service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:02.151814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704c8f2-853b-4c28-979e-f721ed1f8f6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:02.152091 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704c8f2-853b-4c28-979e-f721ed1f8f6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:02.161566 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704c8f2-853b-4c28-979e-f721ed1f8f6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:18:02.163030 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704c8f2-853b-4c28-979e-f721ed1f8f6c service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:18:02.172754 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 726/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:18:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:18:12.085121 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=11 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:18:12.168233 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=14 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:18:22.230631 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a0f7b-f53d-4d89-9759-f9afc112d11a None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:22.231229 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a0f7b-f53d-4d89-9759-f9afc112d11a None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:22.231229 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a0f7b-f53d-4d89-9759-f9afc112d11a None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:22.232604 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 745/1471] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:18:22.238656 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db491283-4ab3-44cb-b876-468152b9bd5c None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:22.239125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db491283-4ab3-44cb-b876-468152b9bd5c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:22.239237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db491283-4ab3-44cb-b876-468152b9bd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:22.256603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-db491283-4ab3-44cb-b876-468152b9bd5c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:18:22.257804 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-db491283-4ab3-44cb-b876-468152b9bd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:18:22.265509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 727/1472] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:18:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:18:22.312790 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e0e30-f91e-4eba-bd9c-e878dcbf8e4d None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:22.313082 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e0e30-f91e-4eba-bd9c-e878dcbf8e4d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:22.313392 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e0e30-f91e-4eba-bd9c-e878dcbf8e4d None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:22.314774 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 746/1473] 10.222.0.52 () {54 vars in 962 bytes} [Thu Jun 23 09:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:18:22.319540 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d602b53-5c65-40a2-8db1-4148e96727bd None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:22.319825 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d602b53-5c65-40a2-8db1-4148e96727bd None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:22.320102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d602b53-5c65-40a2-8db1-4148e96727bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:22.338186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-3d602b53-5c65-40a2-8db1-4148e96727bd None None] MFA Rules not processed for user `36a6a3190dad4948a505bbe4c7095699`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:18:22.339402 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-3d602b53-5c65-40a2-8db1-4148e96727bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:18:22.346307 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 728/1474] 10.222.0.52 () {58 vars in 1059 bytes} [Thu Jun 23 09:18:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:18:22.353464 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e0e30-f91e-4eba-bd9c-e878dcbf8e4d None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:18:22.363967 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e0e30-f91e-4eba-bd9c-e878dcbf8e4d None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:18:22.367393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01561ef8-bcf9-4de1-99b8-3d4c8615c5d3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36a6a3190dad4948a505bbe4c7095699', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:18:22.368019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01561ef8-bcf9-4de1-99b8-3d4c8615c5d3 service glance] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:22.368303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01561ef8-bcf9-4de1-99b8-3d4c8615c5d3 service glance] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:22.368581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01561ef8-bcf9-4de1-99b8-3d4c8615c5d3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:22.377267 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01561ef8-bcf9-4de1-99b8-3d4c8615c5d3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:18:22.378700 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01561ef8-bcf9-4de1-99b8-3d4c8615c5d3 service glance] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:18:22.387831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 747/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:18:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:18:24.503042 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6f5e0-ca28-48fa-b673-2f8863512380 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:24.503556 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6f5e0-ca28-48fa-b673-2f8863512380 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:24.503556 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6f5e0-ca28-48fa-b673-2f8863512380 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:24.504884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 729/1476] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:18:24.510145 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b8825-8056-42cf-8ba9-e7b4b311aef8 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:24.510562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b8825-8056-42cf-8ba9-e7b4b311aef8 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:24.510697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b8825-8056-42cf-8ba9-e7b4b311aef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:24.528997 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-bf9b8825-8056-42cf-8ba9-e7b4b311aef8 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:18:24.530184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-bf9b8825-8056-42cf-8ba9-e7b4b311aef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:18:24.538538 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 748/1477] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:18:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:18:24.561761 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6f5e0-ca28-48fa-b673-2f8863512380 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:18:24.574137 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6f5e0-ca28-48fa-b673-2f8863512380 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:18:24.578543 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f122eff-4984-4e6e-8814-1746eaa637f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:18:24.579793 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f122eff-4984-4e6e-8814-1746eaa637f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:24.580080 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f122eff-4984-4e6e-8814-1746eaa637f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:24.580361 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f122eff-4984-4e6e-8814-1746eaa637f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:24.581115 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f122eff-4984-4e6e-8814-1746eaa637f2 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:18:24.582842 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: WARNING py.warnings [None req-6f122eff-4984-4e6e-8814-1746eaa637f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 09:18:24.582842 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: warnings.warn(msg) Jun 23 09:18:24.582842 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: Jun 23 09:18:24.584039 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f122eff-4984-4e6e-8814-1746eaa637f2 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:18:24.603434 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 730/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jun 23 09:18:24 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:18:26.663229 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6336921-b76f-44e8-a9e1-2ef83f501af2 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:26.663792 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6336921-b76f-44e8-a9e1-2ef83f501af2 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:26.663792 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6336921-b76f-44e8-a9e1-2ef83f501af2 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:26.665175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 749/1479] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:18:26.670452 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da512c5-42bb-435e-baa0-1acecaccadda None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:26.670897 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da512c5-42bb-435e-baa0-1acecaccadda None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:26.671067 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da512c5-42bb-435e-baa0-1acecaccadda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:26.689808 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-0da512c5-42bb-435e-baa0-1acecaccadda None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:18:26.691857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-0da512c5-42bb-435e-baa0-1acecaccadda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:18:26.699777 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 731/1480] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:18:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:18:26.746194 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6336921-b76f-44e8-a9e1-2ef83f501af2 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:18:26.756840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6336921-b76f-44e8-a9e1-2ef83f501af2 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:18:26.762066 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620da9b-57b1-44dd-8db0-3da958334ce8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:18:26.762699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620da9b-57b1-44dd-8db0-3da958334ce8 service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:26.763030 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620da9b-57b1-44dd-8db0-3da958334ce8 service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:26.763312 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620da9b-57b1-44dd-8db0-3da958334ce8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:26.771283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620da9b-57b1-44dd-8db0-3da958334ce8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:18:26.772861 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620da9b-57b1-44dd-8db0-3da958334ce8 service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:18:26.782457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 750/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:18:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:18:28.969546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37c1c1-d090-4971-88db-b595030a7c9d None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:28.970129 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37c1c1-d090-4971-88db-b595030a7c9d None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:28.970129 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37c1c1-d090-4971-88db-b595030a7c9d None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:28.971538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 732/1482] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:18:28.977375 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99241e-91eb-4d14-9a7c-f25ea69d4cd7 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:28.977816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99241e-91eb-4d14-9a7c-f25ea69d4cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:28.977935 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99241e-91eb-4d14-9a7c-f25ea69d4cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:28.995408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-bc99241e-91eb-4d14-9a7c-f25ea69d4cd7 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:18:28.996628 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-bc99241e-91eb-4d14-9a7c-f25ea69d4cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:18:29.004478 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 751/1483] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:18:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:18:29.044531 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37c1c1-d090-4971-88db-b595030a7c9d None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:18:29.055085 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37c1c1-d090-4971-88db-b595030a7c9d None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:18:29.058600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd181e-8ea9-4378-a805-768c840875e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:18:29.059254 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd181e-8ea9-4378-a805-768c840875e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:29.072265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd181e-8ea9-4378-a805-768c840875e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:29.072265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd181e-8ea9-4378-a805-768c840875e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:29.072265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd181e-8ea9-4378-a805-768c840875e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:18:29.072265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd181e-8ea9-4378-a805-768c840875e6 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:18:29.079749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 733/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:18:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:18:31.148599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ff510-ae99-4578-8865-0e4563e43d66 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:31.149140 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ff510-ae99-4578-8865-0e4563e43d66 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:31.149242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ff510-ae99-4578-8865-0e4563e43d66 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:31.150619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 752/1485] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:18:31.155940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdd1ec-b2ac-4839-9ff7-5fe37c499311 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:31.156329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdd1ec-b2ac-4839-9ff7-5fe37c499311 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:31.156503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdd1ec-b2ac-4839-9ff7-5fe37c499311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:31.174396 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-0ffdd1ec-b2ac-4839-9ff7-5fe37c499311 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:18:31.175630 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-0ffdd1ec-b2ac-4839-9ff7-5fe37c499311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:18:31.183459 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 734/1486] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:18:31.221084 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ff510-ae99-4578-8865-0e4563e43d66 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:18:31.231843 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ff510-ae99-4578-8865-0e4563e43d66 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:18:31.235796 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31cbc0-205a-46d9-8bc4-fa09b76de5b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:18:31.236436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31cbc0-205a-46d9-8bc4-fa09b76de5b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:31.236722 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31cbc0-205a-46d9-8bc4-fa09b76de5b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:31.236999 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31cbc0-205a-46d9-8bc4-fa09b76de5b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:31.245716 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31cbc0-205a-46d9-8bc4-fa09b76de5b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:18:31.247168 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31cbc0-205a-46d9-8bc4-fa09b76de5b8 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:18:31.256882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 753/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:18:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:18:33.387371 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939d9c6-8177-4424-8a0f-ceb64093dec6 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:33.387940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939d9c6-8177-4424-8a0f-ceb64093dec6 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:33.387940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939d9c6-8177-4424-8a0f-ceb64093dec6 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:33.389339 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 735/1488] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:18:33.395154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d7521-7ef5-41dd-8459-7e305d05344e None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:33.395601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d7521-7ef5-41dd-8459-7e305d05344e None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:33.395755 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d7521-7ef5-41dd-8459-7e305d05344e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:33.413670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-dd3d7521-7ef5-41dd-8459-7e305d05344e None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:18:33.414869 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-dd3d7521-7ef5-41dd-8459-7e305d05344e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:18:33.422783 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 754/1489] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:18:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:18:33.463422 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939d9c6-8177-4424-8a0f-ceb64093dec6 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:18:33.465107 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d939d9c6-8177-4424-8a0f-ceb64093dec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:18:33.478740 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939d9c6-8177-4424-8a0f-ceb64093dec6 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:18:33.482254 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0132a82-3528-4b92-ad44-6fab9b9bd873 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:18:33.482893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0132a82-3528-4b92-ad44-6fab9b9bd873 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:33.483209 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0132a82-3528-4b92-ad44-6fab9b9bd873 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:33.483488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0132a82-3528-4b92-ad44-6fab9b9bd873 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:33.492302 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0132a82-3528-4b92-ad44-6fab9b9bd873 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:18:33.493733 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0132a82-3528-4b92-ad44-6fab9b9bd873 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:18:33.503063 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 736/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:18:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:18:36.092151 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2e926-c95e-4a9c-9976-d045a4336c0c None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:36.092707 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2e926-c95e-4a9c-9976-d045a4336c0c None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:36.092707 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2e926-c95e-4a9c-9976-d045a4336c0c None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:36.094077 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 755/1491] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jun 23 09:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:18:36.099540 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ff24a-2da8-4310-8cce-4f1e5e24fdf1 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:36.100018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ff24a-2da8-4310-8cce-4f1e5e24fdf1 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:36.100116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ff24a-2da8-4310-8cce-4f1e5e24fdf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:36.117420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-912ff24a-2da8-4310-8cce-4f1e5e24fdf1 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:18:36.118616 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-912ff24a-2da8-4310-8cce-4f1e5e24fdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:18:36.126989 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 737/1492] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jun 23 09:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:18:36.165780 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2e926-c95e-4a9c-9976-d045a4336c0c None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:18:36.176709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2e926-c95e-4a9c-9976-d045a4336c0c None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:18:36.180315 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e98e3-8a3a-4b8a-8bd1-1abc027e60f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:18:36.180962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e98e3-8a3a-4b8a-8bd1-1abc027e60f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:18:36.181279 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e98e3-8a3a-4b8a-8bd1-1abc027e60f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:18:36.181559 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e98e3-8a3a-4b8a-8bd1-1abc027e60f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:18:36.191115 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e98e3-8a3a-4b8a-8bd1-1abc027e60f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:18:36.192547 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e98e3-8a3a-4b8a-8bd1-1abc027e60f0 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:18:36.201931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 756/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:18:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:18:46.112420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=33 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:18:46.197807 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=33 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:20:10.094233 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1caaec-905e-40ed-86b2-24172eccc18b None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:20:10.094869 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1caaec-905e-40ed-86b2-24172eccc18b None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:20:10.094869 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1caaec-905e-40ed-86b2-24172eccc18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:20:10.115200 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-cb1caaec-905e-40ed-86b2-24172eccc18b None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:20:10.116406 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-cb1caaec-905e-40ed-86b2-24172eccc18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:20:10.124216 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 738/1494] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:20:10.134011 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f678273-6f37-4299-89cb-fa3f0eab6e34 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:20:10.134510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f678273-6f37-4299-89cb-fa3f0eab6e34 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:20:10.134621 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f678273-6f37-4299-89cb-fa3f0eab6e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:20:10.153024 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-3f678273-6f37-4299-89cb-fa3f0eab6e34 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:20:10.154213 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-3f678273-6f37-4299-89cb-fa3f0eab6e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:20:10.162146 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 757/1495] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:20:10.166621 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1caaec-905e-40ed-86b2-24172eccc18b None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:20:10.176852 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1caaec-905e-40ed-86b2-24172eccc18b None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:20:10.181328 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354c947-2d53-4057-91a7-e5be8506928d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:20:10.181984 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354c947-2d53-4057-91a7-e5be8506928d admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:20:10.182271 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354c947-2d53-4057-91a7-e5be8506928d admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:20:10.182550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354c947-2d53-4057-91a7-e5be8506928d admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:20:10.183314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d354c947-2d53-4057-91a7-e5be8506928d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:20:10.183624 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d354c947-2d53-4057-91a7-e5be8506928d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:20:10.184638 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: WARNING py.warnings [None req-d354c947-2d53-4057-91a7-e5be8506928d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 09:20:10.184638 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: warnings.warn(msg) Jun 23 09:20:10.184638 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: Jun 23 09:20:10.185527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d354c947-2d53-4057-91a7-e5be8506928d admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:20:10.197393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 739/1496] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:20:10.205349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f678273-6f37-4299-89cb-fa3f0eab6e34 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:20:10.214531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f678273-6f37-4299-89cb-fa3f0eab6e34 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:20:10.219107 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0360251-8d6f-4397-83a2-6e9355c622b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:20:10.219762 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0360251-8d6f-4397-83a2-6e9355c622b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:20:10.220052 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0360251-8d6f-4397-83a2-6e9355c622b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:20:10.220329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0360251-8d6f-4397-83a2-6e9355c622b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:20:10.221016 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0360251-8d6f-4397-83a2-6e9355c622b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:20:10.222787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: WARNING py.warnings [None req-e0360251-8d6f-4397-83a2-6e9355c622b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 09:20:10.222787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: warnings.warn(msg) Jun 23 09:20:10.222787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: Jun 23 09:20:10.224027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0360251-8d6f-4397-83a2-6e9355c622b8 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:20:10.344863 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 758/1497] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:20:10 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:20:10.349135 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354c947-2d53-4057-91a7-e5be8506928d admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:20:10.359982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354c947-2d53-4057-91a7-e5be8506928d admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:20:10.415790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4794e5-d06a-483f-a962-f91bce881d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:20:10.416509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4794e5-d06a-483f-a962-f91bce881d95 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:20:10.416804 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4794e5-d06a-483f-a962-f91bce881d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:20:10.417082 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4794e5-d06a-483f-a962-f91bce881d95 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:20:10.417811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4794e5-d06a-483f-a962-f91bce881d95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:20:10.418836 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: WARNING py.warnings [None req-5d4794e5-d06a-483f-a962-f91bce881d95 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 09:20:10.418836 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: warnings.warn(msg) Jun 23 09:20:10.418836 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: Jun 23 09:20:10.420088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4794e5-d06a-483f-a962-f91bce881d95 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:20:10.552158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 740/1498] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:20:10 2022] POST /identity/v3/users => generated 290 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:20:10.556384 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0360251-8d6f-4397-83a2-6e9355c622b8 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:20:10.566622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0360251-8d6f-4397-83a2-6e9355c622b8 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:20:10.571174 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7a95e-35d2-42c4-a8c2-ecf6c1757fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:20:10.571842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7a95e-35d2-42c4-a8c2-ecf6c1757fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:20:10.572135 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7a95e-35d2-42c4-a8c2-ecf6c1757fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:20:10.572420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7a95e-35d2-42c4-a8c2-ecf6c1757fd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:20:10.573151 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7a95e-35d2-42c4-a8c2-ecf6c1757fd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:20:10.574164 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: WARNING py.warnings [None req-1cf7a95e-35d2-42c4-a8c2-ecf6c1757fd7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 09:20:10.574164 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: warnings.warn(msg) Jun 23 09:20:10.574164 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: Jun 23 09:20:10.575083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7a95e-35d2-42c4-a8c2-ecf6c1757fd7 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:20:10.587627 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 759/1499] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:20:10.591563 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4794e5-d06a-483f-a962-f91bce881d95 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:20:10.602498 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4794e5-d06a-483f-a962-f91bce881d95 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:20:10.607148 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58dd79-106b-4e46-8d3d-fb2a48cce802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:20:10.607807 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58dd79-106b-4e46-8d3d-fb2a48cce802 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:20:10.608098 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58dd79-106b-4e46-8d3d-fb2a48cce802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:20:10.608377 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58dd79-106b-4e46-8d3d-fb2a48cce802 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:20:10.609726 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58dd79-106b-4e46-8d3d-fb2a48cce802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:20:10.610783 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: WARNING py.warnings [None req-ae58dd79-106b-4e46-8d3d-fb2a48cce802 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 09:20:10.610783 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: warnings.warn(msg) Jun 23 09:20:10.610783 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: Jun 23 09:20:10.611703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58dd79-106b-4e46-8d3d-fb2a48cce802 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:20:10.625379 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 741/1500] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:20:10.629477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7a95e-35d2-42c4-a8c2-ecf6c1757fd7 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:20:10.639428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7a95e-35d2-42c4-a8c2-ecf6c1757fd7 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:20:10.644167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caacf0c-ae70-4d20-a633-581fcf1411c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:20:10.645073 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caacf0c-ae70-4d20-a633-581fcf1411c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:20:10.645362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caacf0c-ae70-4d20-a633-581fcf1411c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:20:10.645639 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caacf0c-ae70-4d20-a633-581fcf1411c7 admin admin] PATH_INFO: `/v3/projects/de8ee20c6fcf4d56913d0ecd5e00456e/users/09ab298b0f3e41b1a3a92f9cff7b4d80/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:20:10.660548 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caacf0c-ae70-4d20-a633-581fcf1411c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de8ee20c6fcf4d56913d0ecd5e00456e, user_id=09ab298b0f3e41b1a3a92f9cff7b4d80, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:20:10.661559 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: WARNING py.warnings [None req-9caacf0c-ae70-4d20-a633-581fcf1411c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 09:20:10.661559 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: warnings.warn(msg) Jun 23 09:20:10.661559 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: Jun 23 09:20:10.663514 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caacf0c-ae70-4d20-a633-581fcf1411c7 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:20:10.772226 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 760/1501] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:20:10 2022] PUT /identity/v3/projects/de8ee20c6fcf4d56913d0ecd5e00456e/users/09ab298b0f3e41b1a3a92f9cff7b4d80/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:20:10.783981 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341dafc-5046-4762-b755-9421dbccaed6 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:20:10.784271 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341dafc-5046-4762-b755-9421dbccaed6 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:20:10.784550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341dafc-5046-4762-b755-9421dbccaed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:20:10.810273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-7341dafc-5046-4762-b755-9421dbccaed6 None None] MFA Rules not processed for user `09ab298b0f3e41b1a3a92f9cff7b4d80`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:20:10.811553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-7341dafc-5046-4762-b755-9421dbccaed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:20:10.873040 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 742/1502] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:20:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:20:10.886039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3316ec-3c60-40e8-854a-15513bd35f81 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:20:10.886338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3316ec-3c60-40e8-854a-15513bd35f81 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:20:10.886650 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3316ec-3c60-40e8-854a-15513bd35f81 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:20:10.888048 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 761/1503] 10.222.0.52 () {54 vars in 963 bytes} [Thu Jun 23 09:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:20:10.893590 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c831c-e308-46e5-9248-82d7166824b3 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:20:10.893912 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c831c-e308-46e5-9248-82d7166824b3 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:20:10.894201 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c831c-e308-46e5-9248-82d7166824b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:20:10.912086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-ba1c831c-e308-46e5-9248-82d7166824b3 None None] MFA Rules not processed for user `c21a758d0e2645318c89a4fe8805df0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:20:10.913262 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ba1c831c-e308-46e5-9248-82d7166824b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:20:10.954823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:20:10.961803 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 743/1504] 10.222.0.52 () {58 vars in 1060 bytes} [Thu Jun 23 09:20:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:20:10.969975 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3316ec-3c60-40e8-854a-15513bd35f81 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:20:10.980728 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3316ec-3c60-40e8-854a-15513bd35f81 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:20:10.984224 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb358a-8f53-4169-aa51-75eb12ec41c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:20:10.984855 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb358a-8f53-4169-aa51-75eb12ec41c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:20:10.985142 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb358a-8f53-4169-aa51-75eb12ec41c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:20:10.985422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb358a-8f53-4169-aa51-75eb12ec41c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:20:10.993798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb358a-8f53-4169-aa51-75eb12ec41c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:20:10.995266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb358a-8f53-4169-aa51-75eb12ec41c1 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:20:11.004173 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 762/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:20:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:20:11.051655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f2f3d7-35e8-405c-94c3-759b16b9bbe1 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:20:11.051949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f2f3d7-35e8-405c-94c3-759b16b9bbe1 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:20:11.052228 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f2f3d7-35e8-405c-94c3-759b16b9bbe1 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:20:11.053511 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 744/1506] 10.222.0.52 () {52 vars in 829 bytes} [Thu Jun 23 09:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:20:11.057117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb358a-8f53-4169-aa51-75eb12ec41c1 service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:20:11.129926 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb358a-8f53-4169-aa51-75eb12ec41c1 service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:20:11.138268 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b38ea-9137-4db8-b50c-fc12aaa022c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:20:11.139257 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b38ea-9137-4db8-b50c-fc12aaa022c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:20:11.139611 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b38ea-9137-4db8-b50c-fc12aaa022c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:20:11.139966 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b38ea-9137-4db8-b50c-fc12aaa022c9 admin admin] PATH_INFO: `/v3/users/09ab298b0f3e41b1a3a92f9cff7b4d80` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:20:11.141736 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b38ea-9137-4db8-b50c-fc12aaa022c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09ab298b0f3e41b1a3a92f9cff7b4d80)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:20:11.142818 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: WARNING py.warnings [None req-2b5b38ea-9137-4db8-b50c-fc12aaa022c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 09:20:11.142818 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: warnings.warn(msg) Jun 23 09:20:11.142818 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: Jun 23 09:20:11.144229 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b38ea-9137-4db8-b50c-fc12aaa022c9 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:20:11.501678 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-2b5b38ea-9137-4db8-b50c-fc12aaa022c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09ab298b0f3e41b1a3a92f9cff7b4d80'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:20:11.513858 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-2b5b38ea-9137-4db8-b50c-fc12aaa022c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09ab298b0f3e41b1a3a92f9cff7b4d80'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:20:11.514634 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-2b5b38ea-9137-4db8-b50c-fc12aaa022c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09ab298b0f3e41b1a3a92f9cff7b4d80'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:20:11.515223 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-2b5b38ea-9137-4db8-b50c-fc12aaa022c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09ab298b0f3e41b1a3a92f9cff7b4d80'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:20:11.533236 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-2b5b38ea-9137-4db8-b50c-fc12aaa022c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09ab298b0f3e41b1a3a92f9cff7b4d80'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:20:11.537328 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:20:11.740207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 763/1507] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:20:11 2022] DELETE /identity/v3/users/09ab298b0f3e41b1a3a92f9cff7b4d80 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:20:11.744685 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55d21c-de54-4b10-bbba-0fc86d84962e None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:20:11.745143 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55d21c-de54-4b10-bbba-0fc86d84962e None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:20:11.745259 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55d21c-de54-4b10-bbba-0fc86d84962e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:20:11.769231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-bd55d21c-de54-4b10-bbba-0fc86d84962e None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:20:11.769231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-bd55d21c-de54-4b10-bbba-0fc86d84962e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:20:11.826022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 745/1508] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:20:11.836188 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b38ea-9137-4db8-b50c-fc12aaa022c9 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:20:11.837618 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-2b5b38ea-9137-4db8-b50c-fc12aaa022c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:20:11.894215 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b38ea-9137-4db8-b50c-fc12aaa022c9 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:20:11.897752 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f6b6a-31fe-4e81-b80e-8a936d81ee7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:20:11.898398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f6b6a-31fe-4e81-b80e-8a936d81ee7a service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:20:11.898683 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f6b6a-31fe-4e81-b80e-8a936d81ee7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:20:11.898993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f6b6a-31fe-4e81-b80e-8a936d81ee7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:20:11.908445 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f6b6a-31fe-4e81-b80e-8a936d81ee7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:20:11.909870 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f6b6a-31fe-4e81-b80e-8a936d81ee7a service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:20:11.919092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 764/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:20:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:20:12.430941 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55d21c-de54-4b10-bbba-0fc86d84962e None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:20:12.432433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-bd55d21c-de54-4b10-bbba-0fc86d84962e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:20:12.444971 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55d21c-de54-4b10-bbba-0fc86d84962e None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:20:12.454473 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2297770-45f7-40ee-83dc-3b796bdf6808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:20:12.454473 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2297770-45f7-40ee-83dc-3b796bdf6808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:20:12.454473 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2297770-45f7-40ee-83dc-3b796bdf6808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:20:12.454473 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2297770-45f7-40ee-83dc-3b796bdf6808 admin admin] PATH_INFO: `/v3/projects/de8ee20c6fcf4d56913d0ecd5e00456e` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:20:12.455234 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2297770-45f7-40ee-83dc-3b796bdf6808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de8ee20c6fcf4d56913d0ecd5e00456e)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:20:12.455234 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: WARNING py.warnings [None req-f2297770-45f7-40ee-83dc-3b796bdf6808 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 09:20:12.455234 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: warnings.warn(msg) Jun 23 09:20:12.455234 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: Jun 23 09:20:12.455234 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2297770-45f7-40ee-83dc-3b796bdf6808 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:20:12.589707 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f2297770-45f7-40ee-83dc-3b796bdf6808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de8ee20c6fcf4d56913d0ecd5e00456e', 'request_id': 'req-f2297770-45f7-40ee-83dc-3b796bdf6808'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:20:12.603210 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f2297770-45f7-40ee-83dc-3b796bdf6808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de8ee20c6fcf4d56913d0ecd5e00456e', 'request_id': 'req-f2297770-45f7-40ee-83dc-3b796bdf6808'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:20:12.691824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f2297770-45f7-40ee-83dc-3b796bdf6808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de8ee20c6fcf4d56913d0ecd5e00456e', 'request_id': 'req-f2297770-45f7-40ee-83dc-3b796bdf6808'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:20:12.706369 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f2297770-45f7-40ee-83dc-3b796bdf6808 admin admin] The token cache is being invalidate because project de8ee20c6fcf4d56913d0ecd5e00456e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:20:12.706658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f2297770-45f7-40ee-83dc-3b796bdf6808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:20:12.707328 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f2297770-45f7-40ee-83dc-3b796bdf6808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:20:12.709017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 746/1510] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:20:12 2022] DELETE /identity/v3/projects/de8ee20c6fcf4d56913d0ecd5e00456e => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:20:21.915038 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:20:22.701757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:21:51.523256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e60c0-0762-4af2-aa00-aebac4fca9c2 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:21:51.523832 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e60c0-0762-4af2-aa00-aebac4fca9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:21:51.523832 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e60c0-0762-4af2-aa00-aebac4fca9c2 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:21:51.525132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 09:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:21:51.529907 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9f4dd-c3e6-4a28-9172-14f21e9f2e67 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:21:51.530348 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9f4dd-c3e6-4a28-9172-14f21e9f2e67 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:21:51.530502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9f4dd-c3e6-4a28-9172-14f21e9f2e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:21:51.550453 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-7cc9f4dd-c3e6-4a28-9172-14f21e9f2e67 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:21:51.551680 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-7cc9f4dd-c3e6-4a28-9172-14f21e9f2e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:21:51.609418 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 09:21:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:21:51.766012 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e60c0-0762-4af2-aa00-aebac4fca9c2 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:21:51.767849 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-486e60c0-0762-4af2-aa00-aebac4fca9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:21:51.841026 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e60c0-0762-4af2-aa00-aebac4fca9c2 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:21:51.846493 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfde065-a894-4b2b-bc01-f748e7e271a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:21:51.847175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfde065-a894-4b2b-bc01-f748e7e271a3 service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:21:51.847470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfde065-a894-4b2b-bc01-f748e7e271a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:21:51.847754 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfde065-a894-4b2b-bc01-f748e7e271a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:21:51.856063 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfde065-a894-4b2b-bc01-f748e7e271a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:21:51.857667 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfde065-a894-4b2b-bc01-f748e7e271a3 service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:21:51.866961 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 766/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:21:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:21:52.694560 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d2cf8-d632-4b9a-b2aa-a980540affa4 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:21:52.695158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d2cf8-d632-4b9a-b2aa-a980540affa4 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:21:52.695158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d2cf8-d632-4b9a-b2aa-a980540affa4 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:21:52.696516 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 09:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:21:52.701357 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b517f-8e2c-4df8-8690-903a868fa323 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:21:52.702338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b517f-8e2c-4df8-8690-903a868fa323 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:21:52.702535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b517f-8e2c-4df8-8690-903a868fa323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:21:52.720851 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-aa7b517f-8e2c-4df8-8690-903a868fa323 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:21:52.722052 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-aa7b517f-8e2c-4df8-8690-903a868fa323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:21:52.730188 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 09:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:21:52.880706 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d2cf8-d632-4b9a-b2aa-a980540affa4 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:21:52.894834 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d2cf8-d632-4b9a-b2aa-a980540affa4 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:21:52.900128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30913a28-77a5-40cf-830b-65297f4c8dc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:21:52.900755 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30913a28-77a5-40cf-830b-65297f4c8dc9 service nova] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:21:52.901041 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30913a28-77a5-40cf-830b-65297f4c8dc9 service nova] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:21:52.901320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30913a28-77a5-40cf-830b-65297f4c8dc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:21:52.909497 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30913a28-77a5-40cf-830b-65297f4c8dc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:21:52.911197 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30913a28-77a5-40cf-830b-65297f4c8dc9 service nova] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:21:52.921343 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 749/1516] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:21:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:21:53.116747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2162124-9602-49fd-a6aa-6078f00d4d72 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:21:53.117052 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2162124-9602-49fd-a6aa-6078f00d4d72 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:21:53.117329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2162124-9602-49fd-a6aa-6078f00d4d72 None None] PATH_INFO: `/v3` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:21:53.118646 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 768/1517] 10.222.0.55 () {54 vars in 925 bytes} [Thu Jun 23 09:21:53 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:21:53.125268 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e4397-cb07-4a3f-bf94-f7d212a17b8e None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:21:53.125592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e4397-cb07-4a3f-bf94-f7d212a17b8e None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:21:53.125875 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e4397-cb07-4a3f-bf94-f7d212a17b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:21:53.143655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-a64e4397-cb07-4a3f-bf94-f7d212a17b8e None None] MFA Rules not processed for user `98a63b20ceca40e6bceb56eda8ae9686`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:21:53.144887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a64e4397-cb07-4a3f-bf94-f7d212a17b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:21:53.206354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 750/1518] 10.222.0.55 () {58 vars in 1015 bytes} [Thu Jun 23 09:21:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:22:02.561337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153e112-4106-4ea7-b583-1cc2954c5260 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:22:02.561860 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153e112-4106-4ea7-b583-1cc2954c5260 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:22:02.561983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153e112-4106-4ea7-b583-1cc2954c5260 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:22:02.563287 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 769/1519] 10.222.0.52 () {54 vars in 960 bytes} [Thu Jun 23 09:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:22:02.568265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478ea4d-4e4e-4b1a-b189-d8f1732f143c None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:22:02.568690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478ea4d-4e4e-4b1a-b189-d8f1732f143c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:22:02.568827 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478ea4d-4e4e-4b1a-b189-d8f1732f143c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:22:02.586896 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-6478ea4d-4e4e-4b1a-b189-d8f1732f143c None None] MFA Rules not processed for user `98a63b20ceca40e6bceb56eda8ae9686`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:22:02.588103 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-6478ea4d-4e4e-4b1a-b189-d8f1732f143c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:22:02.597071 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 751/1520] 10.222.0.52 () {58 vars in 1057 bytes} [Thu Jun 23 09:22:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:22:02.604646 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153e112-4106-4ea7-b583-1cc2954c5260 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:22:02.617545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153e112-4106-4ea7-b583-1cc2954c5260 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:22:02.622368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0214e-f254-40e8-abc9-d7647abd2519 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63b20ceca40e6bceb56eda8ae9686', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:22:02.623025 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0214e-f254-40e8-abc9-d7647abd2519 service placement] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:22:02.623339 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0214e-f254-40e8-abc9-d7647abd2519 service placement] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:22:02.623618 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0214e-f254-40e8-abc9-d7647abd2519 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:22:02.631768 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f0214e-f254-40e8-abc9-d7647abd2519 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:22:02.633342 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f0214e-f254-40e8-abc9-d7647abd2519 service placement] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:22:02.643939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 770/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:22:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:22:04.185774 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f057930-34c7-44af-beec-60049108bdb2 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:22:04.186285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f057930-34c7-44af-beec-60049108bdb2 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:22:04.186397 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f057930-34c7-44af-beec-60049108bdb2 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:22:04.187752 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 752/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 09:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:22:04.192699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65e205-8339-45ed-9ac7-fb8b7a479c1d None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:22:04.193132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65e205-8339-45ed-9ac7-fb8b7a479c1d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:22:04.193332 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65e205-8339-45ed-9ac7-fb8b7a479c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:22:04.213537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-0f65e205-8339-45ed-9ac7-fb8b7a479c1d None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:22:04.214730 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-0f65e205-8339-45ed-9ac7-fb8b7a479c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:22:04.223185 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 771/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 09:22:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:22:04.384517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f057930-34c7-44af-beec-60049108bdb2 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:22:04.402166 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f057930-34c7-44af-beec-60049108bdb2 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:22:04.411791 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c0244-136b-443b-88fc-5a2428828803 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:22:04.412376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c0244-136b-443b-88fc-5a2428828803 service nova] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:22:04.412663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c0244-136b-443b-88fc-5a2428828803 service nova] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:22:04.412945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c0244-136b-443b-88fc-5a2428828803 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:22:04.421559 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c0244-136b-443b-88fc-5a2428828803 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:22:04.423452 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c0244-136b-443b-88fc-5a2428828803 service nova] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:22:04.432940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 753/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:22:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:22:05.255099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35d8c9-07eb-49fd-86d6-19bca1441b21 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:22:05.255652 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35d8c9-07eb-49fd-86d6-19bca1441b21 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:22:05.255769 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35d8c9-07eb-49fd-86d6-19bca1441b21 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:22:05.257091 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 772/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 09:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:22:05.261745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c50cf9-d8a5-4aa8-8bbc-9b6e91fd8570 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:22:05.262187 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c50cf9-d8a5-4aa8-8bbc-9b6e91fd8570 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:22:05.262286 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c50cf9-d8a5-4aa8-8bbc-9b6e91fd8570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:22:05.280335 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-40c50cf9-d8a5-4aa8-8bbc-9b6e91fd8570 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:22:05.281514 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-40c50cf9-d8a5-4aa8-8bbc-9b6e91fd8570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:22:05.289069 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 754/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 09:22:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:22:05.443040 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35d8c9-07eb-49fd-86d6-19bca1441b21 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:22:05.457126 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35d8c9-07eb-49fd-86d6-19bca1441b21 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:22:05.462400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41104b3d-30d6-4e02-ad42-fc00e55d19ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:22:05.463081 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41104b3d-30d6-4e02-ad42-fc00e55d19ec service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:22:05.463375 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41104b3d-30d6-4e02-ad42-fc00e55d19ec service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:22:05.463681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41104b3d-30d6-4e02-ad42-fc00e55d19ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:22:05.472591 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41104b3d-30d6-4e02-ad42-fc00e55d19ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:22:05.474204 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41104b3d-30d6-4e02-ad42-fc00e55d19ec service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:22:05.484457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 773/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:22:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:22:09.543537 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd9236-7f6c-4819-a8ac-deb4c67a6f3e None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:22:09.544098 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd9236-7f6c-4819-a8ac-deb4c67a6f3e None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:22:09.544098 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd9236-7f6c-4819-a8ac-deb4c67a6f3e None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:22:09.545497 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 755/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 09:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:22:09.551233 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45258c8f-4aad-4687-8984-a2d73f986a7b None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:22:09.551647 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45258c8f-4aad-4687-8984-a2d73f986a7b None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:22:09.551787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45258c8f-4aad-4687-8984-a2d73f986a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:22:09.569265 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-45258c8f-4aad-4687-8984-a2d73f986a7b None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:22:09.570448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-45258c8f-4aad-4687-8984-a2d73f986a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:22:09.578638 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 774/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 09:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:22:09.615532 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd9236-7f6c-4819-a8ac-deb4c67a6f3e None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:22:09.617241 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-81fd9236-7f6c-4819-a8ac-deb4c67a6f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:22:09.674428 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd9236-7f6c-4819-a8ac-deb4c67a6f3e None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:22:09.677953 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da04bda-392e-4678-a898-f78c2e794ed5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:22:09.678592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da04bda-392e-4678-a898-f78c2e794ed5 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:22:09.678879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da04bda-392e-4678-a898-f78c2e794ed5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:22:09.679191 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da04bda-392e-4678-a898-f78c2e794ed5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:22:09.688108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da04bda-392e-4678-a898-f78c2e794ed5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:22:09.689615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da04bda-392e-4678-a898-f78c2e794ed5 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:22:09.694623 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:22:09.699409 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 756/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:22:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:22:11.731648 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf151ff-bb02-4753-85d2-e213c6e2f586 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:22:11.732220 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf151ff-bb02-4753-85d2-e213c6e2f586 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:22:11.732220 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf151ff-bb02-4753-85d2-e213c6e2f586 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:22:11.733793 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 775/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 09:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:22:11.738934 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ea7ff-83b4-4934-8dcd-a5664247e8dd None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:22:11.739368 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ea7ff-83b4-4934-8dcd-a5664247e8dd None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:22:11.739602 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ea7ff-83b4-4934-8dcd-a5664247e8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:22:11.757878 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-675ea7ff-83b4-4934-8dcd-a5664247e8dd None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:22:11.759211 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-675ea7ff-83b4-4934-8dcd-a5664247e8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:22:11.767808 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 757/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 09:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:22:11.803510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf151ff-bb02-4753-85d2-e213c6e2f586 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:22:11.814350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf151ff-bb02-4753-85d2-e213c6e2f586 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:22:11.817842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbd4874-9e4c-47e3-84fb-3ff61835bba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:22:11.818488 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd4874-9e4c-47e3-84fb-3ff61835bba1 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:22:11.818775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd4874-9e4c-47e3-84fb-3ff61835bba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:22:11.819079 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd4874-9e4c-47e3-84fb-3ff61835bba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:22:11.828352 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbd4874-9e4c-47e3-84fb-3ff61835bba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:22:11.829788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbd4874-9e4c-47e3-84fb-3ff61835bba1 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:22:11.839097 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 776/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:22:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:22:15.412447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f0442-1bb6-4b63-afc4-d98c19bbb7c2 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:22:15.413015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f0442-1bb6-4b63-afc4-d98c19bbb7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:22:15.413125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f0442-1bb6-4b63-afc4-d98c19bbb7c2 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:22:15.414448 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 758/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 09:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:22:15.419834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cfd7d-1478-47a5-b4d8-40d1a8d99459 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:22:15.420731 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cfd7d-1478-47a5-b4d8-40d1a8d99459 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:22:15.420911 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cfd7d-1478-47a5-b4d8-40d1a8d99459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:22:15.427566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=62 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:22:15.443777 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-eb0cfd7d-1478-47a5-b4d8-40d1a8d99459 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:22:15.444996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-eb0cfd7d-1478-47a5-b4d8-40d1a8d99459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:22:15.452779 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 777/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 09:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:22:15.762057 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f0442-1bb6-4b63-afc4-d98c19bbb7c2 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:22:15.763769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-519f0442-1bb6-4b63-afc4-d98c19bbb7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:22:15.819502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f0442-1bb6-4b63-afc4-d98c19bbb7c2 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:22:15.822987 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef871666-ddf4-40da-bc08-2c1b1a1a5e68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:22:15.823621 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef871666-ddf4-40da-bc08-2c1b1a1a5e68 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:22:15.823905 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef871666-ddf4-40da-bc08-2c1b1a1a5e68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:22:15.824183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef871666-ddf4-40da-bc08-2c1b1a1a5e68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:22:15.832928 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef871666-ddf4-40da-bc08-2c1b1a1a5e68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:22:15.834371 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef871666-ddf4-40da-bc08-2c1b1a1a5e68 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:22:15.847538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 759/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:22:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:22:17.925817 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a021ecf-71a5-4a23-b095-6480194ad5f1 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:22:17.926470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a021ecf-71a5-4a23-b095-6480194ad5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:22:17.926470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a021ecf-71a5-4a23-b095-6480194ad5f1 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:22:17.927855 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 778/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 09:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:22:17.933193 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d47036-6773-4c7e-8172-ba3486ddf726 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:22:17.933642 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d47036-6773-4c7e-8172-ba3486ddf726 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:22:17.933797 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d47036-6773-4c7e-8172-ba3486ddf726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:22:17.952147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-31d47036-6773-4c7e-8172-ba3486ddf726 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:22:17.956808 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-31d47036-6773-4c7e-8172-ba3486ddf726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:22:17.961270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 760/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 09:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:22:18.265343 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a021ecf-71a5-4a23-b095-6480194ad5f1 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:22:18.267096 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-5a021ecf-71a5-4a23-b095-6480194ad5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:22:18.280422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a021ecf-71a5-4a23-b095-6480194ad5f1 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:22:18.284241 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2087fbb-5eb2-42de-9fd5-261887565d34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:22:18.284882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2087fbb-5eb2-42de-9fd5-261887565d34 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:22:18.285168 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2087fbb-5eb2-42de-9fd5-261887565d34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:22:18.285462 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2087fbb-5eb2-42de-9fd5-261887565d34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:22:18.294582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2087fbb-5eb2-42de-9fd5-261887565d34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:22:18.296660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2087fbb-5eb2-42de-9fd5-261887565d34 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:22:18.306303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 779/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:22:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:22:27.946820 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=32 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:22:28.301985 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=8 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:46.771037 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfea8cf-d179-437a-abd5-9258d8f70899 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:46.772232 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfea8cf-d179-437a-abd5-9258d8f70899 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:46.774116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfea8cf-d179-437a-abd5-9258d8f70899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:46.795270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-dcfea8cf-d179-437a-abd5-9258d8f70899 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:46.797018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-dcfea8cf-d179-437a-abd5-9258d8f70899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:46.806364 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 761/1540] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:46.829137 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49418680-6dba-4daf-97d7-f5728f9f4b2f None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:46.830180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49418680-6dba-4daf-97d7-f5728f9f4b2f None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:46.830697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49418680-6dba-4daf-97d7-f5728f9f4b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:46.858155 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-49418680-6dba-4daf-97d7-f5728f9f4b2f None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:46.859765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-49418680-6dba-4daf-97d7-f5728f9f4b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:46.868375 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 780/1541] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:46.873279 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfea8cf-d179-437a-abd5-9258d8f70899 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:46.886658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfea8cf-d179-437a-abd5-9258d8f70899 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:46.892385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd61aa3-5c7a-4142-8104-07f4be7da7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:46.893357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd61aa3-5c7a-4142-8104-07f4be7da7e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:46.893876 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd61aa3-5c7a-4142-8104-07f4be7da7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:46.894373 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd61aa3-5c7a-4142-8104-07f4be7da7e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:46.895356 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd61aa3-5c7a-4142-8104-07f4be7da7e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:23:46.895939 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd61aa3-5c7a-4142-8104-07f4be7da7e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:46.897945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd61aa3-5c7a-4142-8104-07f4be7da7e8 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:46.914735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 762/1542] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:23:46.923920 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49418680-6dba-4daf-97d7-f5728f9f4b2f None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:46.941037 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49418680-6dba-4daf-97d7-f5728f9f4b2f None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:46.951625 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd968689-698b-4139-82f4-30bd8877d759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:46.952314 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd968689-698b-4139-82f4-30bd8877d759 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:46.952608 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd968689-698b-4139-82f4-30bd8877d759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:46.952894 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd968689-698b-4139-82f4-30bd8877d759 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:46.953601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd968689-698b-4139-82f4-30bd8877d759 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:46.955620 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd968689-698b-4139-82f4-30bd8877d759 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:47.060646 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 781/1543] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:23:46 2022] POST /identity/v3/projects => generated 377 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:23:47.065072 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd61aa3-5c7a-4142-8104-07f4be7da7e8 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:47.077865 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd61aa3-5c7a-4142-8104-07f4be7da7e8 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:47.151571 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202ad5a-8af2-4c3b-8935-6103c1c28350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:47.152779 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202ad5a-8af2-4c3b-8935-6103c1c28350 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:47.153258 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202ad5a-8af2-4c3b-8935-6103c1c28350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:47.154278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202ad5a-8af2-4c3b-8935-6103c1c28350 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:47.155360 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1202ad5a-8af2-4c3b-8935-6103c1c28350 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:47.157220 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1202ad5a-8af2-4c3b-8935-6103c1c28350 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:47.282880 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 763/1544] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:23:47 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:47.287175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd968689-698b-4139-82f4-30bd8877d759 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:47.302841 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd968689-698b-4139-82f4-30bd8877d759 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:47.315975 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159fb7c-2cf3-4380-9f9d-1130daa6ee7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:47.316621 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159fb7c-2cf3-4380-9f9d-1130daa6ee7b admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:47.316917 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159fb7c-2cf3-4380-9f9d-1130daa6ee7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:47.317206 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159fb7c-2cf3-4380-9f9d-1130daa6ee7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:47.317997 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159fb7c-2cf3-4380-9f9d-1130daa6ee7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:47.319759 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159fb7c-2cf3-4380-9f9d-1130daa6ee7b admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:47.337482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 782/1545] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:47.342203 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202ad5a-8af2-4c3b-8935-6103c1c28350 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:47.358069 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202ad5a-8af2-4c3b-8935-6103c1c28350 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:47.369068 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83f7f9-c5af-487d-a11e-cabf512b7727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:47.369772 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83f7f9-c5af-487d-a11e-cabf512b7727 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:47.370058 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83f7f9-c5af-487d-a11e-cabf512b7727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:47.370336 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83f7f9-c5af-487d-a11e-cabf512b7727 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:47.371105 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d83f7f9-c5af-487d-a11e-cabf512b7727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:47.373423 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d83f7f9-c5af-487d-a11e-cabf512b7727 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:47.395791 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 764/1546] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:47.404631 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159fb7c-2cf3-4380-9f9d-1130daa6ee7b admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:47.416774 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b4fb5-e06f-4c47-b5a4-f58dc697d243 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:47.417123 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b4fb5-e06f-4c47-b5a4-f58dc697d243 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:47.417346 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b4fb5-e06f-4c47-b5a4-f58dc697d243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:47.417475 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159fb7c-2cf3-4380-9f9d-1130daa6ee7b admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:47.429386 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c59af2-4b61-4cdb-bec5-36d44eade771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:47.430305 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c59af2-4b61-4cdb-bec5-36d44eade771 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:47.430597 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c59af2-4b61-4cdb-bec5-36d44eade771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:47.430884 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c59af2-4b61-4cdb-bec5-36d44eade771 admin admin] PATH_INFO: `/v3/projects/8e9ba71662eb4901b4aeae33eed1ecd7/users/2a31c92089cf47aca10c399dafe6aa55/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:47.458104 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-bd0b4fb5-e06f-4c47-b5a4-f58dc697d243 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:47.459371 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-bd0b4fb5-e06f-4c47-b5a4-f58dc697d243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:47.469324 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c59af2-4b61-4cdb-bec5-36d44eade771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e9ba71662eb4901b4aeae33eed1ecd7, user_id=2a31c92089cf47aca10c399dafe6aa55, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:47.472404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c59af2-4b61-4cdb-bec5-36d44eade771 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:47.476264 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 765/1547] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:47.488322 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18084f2a-163e-47d6-991d-5f903dffb7b7 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:47.494417 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18084f2a-163e-47d6-991d-5f903dffb7b7 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:47.494417 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18084f2a-163e-47d6-991d-5f903dffb7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:47.515186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-18084f2a-163e-47d6-991d-5f903dffb7b7 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:47.516425 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-18084f2a-163e-47d6-991d-5f903dffb7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:47.530758 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 766/1548] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:23:47.532144 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3701b4-7f37-444e-904b-c80c8d4012c5 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:47.532434 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3701b4-7f37-444e-904b-c80c8d4012c5 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:47.532713 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3701b4-7f37-444e-904b-c80c8d4012c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:47.561114 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 783/1549] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:23:47 2022] PUT /identity/v3/projects/8e9ba71662eb4901b4aeae33eed1ecd7/users/2a31c92089cf47aca10c399dafe6aa55/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:23:47.569880 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c59af2-4b61-4cdb-bec5-36d44eade771 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:47.574362 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-eb3701b4-7f37-444e-904b-c80c8d4012c5 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:47.577398 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-eb3701b4-7f37-444e-904b-c80c8d4012c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:47.652362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c59af2-4b61-4cdb-bec5-36d44eade771 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:47.657538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 767/1550] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:47.657687 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e36db4-c687-4e60-94e2-f9f329ead2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:47.658210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e36db4-c687-4e60-94e2-f9f329ead2e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:47.658512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e36db4-c687-4e60-94e2-f9f329ead2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:47.658782 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e36db4-c687-4e60-94e2-f9f329ead2e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:47.659033 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ace8c4-84c7-4e77-8f59-561080cf3b10 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:47.659329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ace8c4-84c7-4e77-8f59-561080cf3b10 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:47.659549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e36db4-c687-4e60-94e2-f9f329ead2e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:23:47.659678 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ace8c4-84c7-4e77-8f59-561080cf3b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:47.659865 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e36db4-c687-4e60-94e2-f9f329ead2e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:47.660890 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: WARNING py.warnings [None req-79e36db4-c687-4e60-94e2-f9f329ead2e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 09:23:47.660890 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: warnings.warn(msg) Jun 23 09:23:47.660890 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: Jun 23 09:23:47.661783 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e36db4-c687-4e60-94e2-f9f329ead2e8 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:47.674079 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 784/1551] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:23:47.675486 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8108184-8b55-45aa-8ce1-5aa857e3a160 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:47.675784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8108184-8b55-45aa-8ce1-5aa857e3a160 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:47.676074 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8108184-8b55-45aa-8ce1-5aa857e3a160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:47.678465 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-a6ace8c4-84c7-4e77-8f59-561080cf3b10 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:47.679719 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a6ace8c4-84c7-4e77-8f59-561080cf3b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:47.687965 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 768/1552] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:47.689226 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10becd6-a21e-4b96-9196-8b0ac52d1d09 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:47.689525 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10becd6-a21e-4b96-9196-8b0ac52d1d09 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:47.689808 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10becd6-a21e-4b96-9196-8b0ac52d1d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:47.706373 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-b8108184-8b55-45aa-8ce1-5aa857e3a160 None None] MFA Rules not processed for user `2a31c92089cf47aca10c399dafe6aa55`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:47.707633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b8108184-8b55-45aa-8ce1-5aa857e3a160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:47.708193 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-b10becd6-a21e-4b96-9196-8b0ac52d1d09 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:47.709356 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-b10becd6-a21e-4b96-9196-8b0ac52d1d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:47.718311 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 769/1553] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:47.719712 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadb695-b812-4618-85e7-94a908b212c1 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:47.720102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadb695-b812-4618-85e7-94a908b212c1 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:47.720285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadb695-b812-4618-85e7-94a908b212c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:47.740468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-ccadb695-b812-4618-85e7-94a908b212c1 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:47.741933 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ccadb695-b812-4618-85e7-94a908b212c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:47.750447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 770/1554] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:47.751702 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5c1ea-77c0-4965-981c-faa1fe1c4150 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:47.752052 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5c1ea-77c0-4965-981c-faa1fe1c4150 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:47.752256 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5c1ea-77c0-4965-981c-faa1fe1c4150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:47.770450 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-5df5c1ea-77c0-4965-981c-faa1fe1c4150 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:47.771681 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5df5c1ea-77c0-4965-981c-faa1fe1c4150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:47.775045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 785/1555] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:47.776429 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e426aae-b5d4-4f44-a60e-37083ec2d492 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:47.776717 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e426aae-b5d4-4f44-a60e-37083ec2d492 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:47.777001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e426aae-b5d4-4f44-a60e-37083ec2d492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:47.781973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 771/1556] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:47.783451 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a38e4-c8c8-49e1-9f83-357e2fd6d786 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:47.783930 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a38e4-c8c8-49e1-9f83-357e2fd6d786 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:47.784384 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a38e4-c8c8-49e1-9f83-357e2fd6d786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:47.795408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-9e426aae-b5d4-4f44-a60e-37083ec2d492 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:47.796671 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-9e426aae-b5d4-4f44-a60e-37083ec2d492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:47.804719 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 786/1557] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:47.805614 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e426aae-b5d4-4f44-a60e-37083ec2d492 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:47.807623 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-670a38e4-c8c8-49e1-9f83-357e2fd6d786 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:47.808795 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-670a38e4-c8c8-49e1-9f83-357e2fd6d786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:47.816680 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 772/1558] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:47.818021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c4436-a3eb-420d-bcfa-06057c5ad555 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:47.818314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c4436-a3eb-420d-bcfa-06057c5ad555 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:47.818445 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e426aae-b5d4-4f44-a60e-37083ec2d492 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:47.818641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c4436-a3eb-420d-bcfa-06057c5ad555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:47.823345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5adf5-cb2a-49f7-9ad8-60b9c65a0444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:47.823953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5adf5-cb2a-49f7-9ad8-60b9c65a0444 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:47.824440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5adf5-cb2a-49f7-9ad8-60b9c65a0444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:47.824769 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5adf5-cb2a-49f7-9ad8-60b9c65a0444 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:47.826882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5adf5-cb2a-49f7-9ad8-60b9c65a0444 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:47.828862 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5adf5-cb2a-49f7-9ad8-60b9c65a0444 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:47.836209 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-9b8c4436-a3eb-420d-bcfa-06057c5ad555 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:47.837494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-9b8c4436-a3eb-420d-bcfa-06057c5ad555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:47.839626 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:47.845972 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 773/1559] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:47.847247 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357447fd-2845-4bfb-866c-fba9c792e35c None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:47.847565 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357447fd-2845-4bfb-866c-fba9c792e35c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:47.847847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357447fd-2845-4bfb-866c-fba9c792e35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:47.861512 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:47.867223 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-357447fd-2845-4bfb-866c-fba9c792e35c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:47.868430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-357447fd-2845-4bfb-866c-fba9c792e35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:47.876849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 774/1560] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:47.878185 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992a6b0-6014-4ce2-88a0-e58c97b4336d None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:47.878599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992a6b0-6014-4ce2-88a0-e58c97b4336d None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:47.878887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992a6b0-6014-4ce2-88a0-e58c97b4336d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:47.896248 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-4992a6b0-6014-4ce2-88a0-e58c97b4336d None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:47.902324 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-4992a6b0-6014-4ce2-88a0-e58c97b4336d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:47.905808 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 775/1561] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:47.906618 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992a6b0-6014-4ce2-88a0-e58c97b4336d None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:47.915804 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992a6b0-6014-4ce2-88a0-e58c97b4336d None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:47.921041 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b63c0d-94c7-4c19-bcd3-6054e4219d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:47.921716 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b63c0d-94c7-4c19-bcd3-6054e4219d24 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:47.922008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b63c0d-94c7-4c19-bcd3-6054e4219d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:47.922287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b63c0d-94c7-4c19-bcd3-6054e4219d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:47.923004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b63c0d-94c7-4c19-bcd3-6054e4219d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:47.924029 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: WARNING py.warnings [None req-b1b63c0d-94c7-4c19-bcd3-6054e4219d24 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 09:23:47.924029 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: warnings.warn(msg) Jun 23 09:23:47.924029 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: Jun 23 09:23:47.925217 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b63c0d-94c7-4c19-bcd3-6054e4219d24 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:47.969266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 787/1562] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:23:47 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:47.970863 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693252f-6ecc-4b78-a0e6-c924614d9636 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:47.971369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693252f-6ecc-4b78-a0e6-c924614d9636 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:47.971811 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693252f-6ecc-4b78-a0e6-c924614d9636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:47.991732 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-1693252f-6ecc-4b78-a0e6-c924614d9636 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:47.992945 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-1693252f-6ecc-4b78-a0e6-c924614d9636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:48.048127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 776/1563] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:23:47 2022] POST /identity/v3/projects => generated 377 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:48.049328 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4077413-44e1-4793-97c3-9de4ba723b22 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:48.049624 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4077413-44e1-4793-97c3-9de4ba723b22 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:48.049913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4077413-44e1-4793-97c3-9de4ba723b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:48.054178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 788/1564] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:48.055167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693252f-6ecc-4b78-a0e6-c924614d9636 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:48.064561 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693252f-6ecc-4b78-a0e6-c924614d9636 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:48.068539 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-a4077413-44e1-4793-97c3-9de4ba723b22 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:48.069746 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a4077413-44e1-4793-97c3-9de4ba723b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:48.120985 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ed790-ca64-4b5d-8cf0-1fe8e7bd058e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:48.121581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ed790-ca64-4b5d-8cf0-1fe8e7bd058e admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:48.121874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ed790-ca64-4b5d-8cf0-1fe8e7bd058e admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:48.122184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ed790-ca64-4b5d-8cf0-1fe8e7bd058e admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:48.122942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ed790-ca64-4b5d-8cf0-1fe8e7bd058e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:23:48.123254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ed790-ca64-4b5d-8cf0-1fe8e7bd058e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:48.124870 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ed790-ca64-4b5d-8cf0-1fe8e7bd058e admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:48.129945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 777/1565] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:48.133597 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4077413-44e1-4793-97c3-9de4ba723b22 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:48.142886 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 789/1566] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:23:48.142886 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ed790-ca64-4b5d-8cf0-1fe8e7bd058e admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:48.146848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ed790-ca64-4b5d-8cf0-1fe8e7bd058e admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:48.148066 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4077413-44e1-4793-97c3-9de4ba723b22 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:48.151504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e085766-5c53-429f-8325-f0b7ca0c91ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:48.152157 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e085766-5c53-429f-8325-f0b7ca0c91ef admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:48.152448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e085766-5c53-429f-8325-f0b7ca0c91ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:48.152737 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e085766-5c53-429f-8325-f0b7ca0c91ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:48.154008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e085766-5c53-429f-8325-f0b7ca0c91ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:23:48.154321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e085766-5c53-429f-8325-f0b7ca0c91ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:48.155946 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e085766-5c53-429f-8325-f0b7ca0c91ef admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:48.156851 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d51ee6-f0cc-4615-b9a3-6b5bf41f43d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:48.157508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d51ee6-f0cc-4615-b9a3-6b5bf41f43d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:48.157792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d51ee6-f0cc-4615-b9a3-6b5bf41f43d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:48.158070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d51ee6-f0cc-4615-b9a3-6b5bf41f43d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:48.158803 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d51ee6-f0cc-4615-b9a3-6b5bf41f43d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:23:48.159139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d51ee6-f0cc-4615-b9a3-6b5bf41f43d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:48.160728 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d51ee6-f0cc-4615-b9a3-6b5bf41f43d6 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:48.175740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 790/1567] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:23:48.175740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e085766-5c53-429f-8325-f0b7ca0c91ef admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:48.176051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 778/1568] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:23:48.176051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d51ee6-f0cc-4615-b9a3-6b5bf41f43d6 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:48.183606 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e085766-5c53-429f-8325-f0b7ca0c91ef admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:48.184546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d51ee6-f0cc-4615-b9a3-6b5bf41f43d6 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:48.188907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf65bab-e1f5-4d36-bcce-7337515ea5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:48.189570 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf65bab-e1f5-4d36-bcce-7337515ea5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:48.189692 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781841b-958b-45b1-8ab7-fed4e9f85b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:48.189860 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf65bab-e1f5-4d36-bcce-7337515ea5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:48.190142 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf65bab-e1f5-4d36-bcce-7337515ea5a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:48.190244 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781841b-958b-45b1-8ab7-fed4e9f85b4d admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:48.190507 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781841b-958b-45b1-8ab7-fed4e9f85b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:48.190784 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781841b-958b-45b1-8ab7-fed4e9f85b4d admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:48.190892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf65bab-e1f5-4d36-bcce-7337515ea5a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:23:48.191214 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf65bab-e1f5-4d36-bcce-7337515ea5a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:48.191499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5781841b-958b-45b1-8ab7-fed4e9f85b4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:48.192819 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf65bab-e1f5-4d36-bcce-7337515ea5a8 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:48.193541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5781841b-958b-45b1-8ab7-fed4e9f85b4d admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:48.204914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 791/1569] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:23:48.205694 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf65bab-e1f5-4d36-bcce-7337515ea5a8 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:48.215311 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf65bab-e1f5-4d36-bcce-7337515ea5a8 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:48.220418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e587b70-8ef7-4c0b-ac9f-4abe873f4fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:48.221065 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e587b70-8ef7-4c0b-ac9f-4abe873f4fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:48.221357 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e587b70-8ef7-4c0b-ac9f-4abe873f4fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:48.221640 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e587b70-8ef7-4c0b-ac9f-4abe873f4fd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:48.222328 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e587b70-8ef7-4c0b-ac9f-4abe873f4fd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:48.223398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: WARNING py.warnings [None req-9e587b70-8ef7-4c0b-ac9f-4abe873f4fd7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 09:23:48.223398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: warnings.warn(msg) Jun 23 09:23:48.223398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: Jun 23 09:23:48.224599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e587b70-8ef7-4c0b-ac9f-4abe873f4fd7 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:48.341775 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 779/1570] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:23:48 2022] POST /identity/v3/users => generated 294 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:48.343335 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781841b-958b-45b1-8ab7-fed4e9f85b4d admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:48.352812 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781841b-958b-45b1-8ab7-fed4e9f85b4d admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:48.357616 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5941e4-d800-4bef-b5d1-6df89e4cee0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:48.358254 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5941e4-d800-4bef-b5d1-6df89e4cee0d admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:48.358544 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5941e4-d800-4bef-b5d1-6df89e4cee0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:48.358825 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5941e4-d800-4bef-b5d1-6df89e4cee0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:48.359584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5941e4-d800-4bef-b5d1-6df89e4cee0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:23:48.359897 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5941e4-d800-4bef-b5d1-6df89e4cee0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:48.361488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5941e4-d800-4bef-b5d1-6df89e4cee0d admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:48.373406 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 780/1571] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:23:48.374102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5941e4-d800-4bef-b5d1-6df89e4cee0d admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:48.383867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5941e4-d800-4bef-b5d1-6df89e4cee0d admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:48.388960 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc65de6-9d52-4f38-a7b5-279a8499868d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:48.389669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc65de6-9d52-4f38-a7b5-279a8499868d admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:48.389869 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc65de6-9d52-4f38-a7b5-279a8499868d admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:48.390229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc65de6-9d52-4f38-a7b5-279a8499868d admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:48.390856 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc65de6-9d52-4f38-a7b5-279a8499868d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:23:48.391197 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc65de6-9d52-4f38-a7b5-279a8499868d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:48.392825 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc65de6-9d52-4f38-a7b5-279a8499868d admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:48.404579 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 781/1572] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:23:48.405312 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc65de6-9d52-4f38-a7b5-279a8499868d admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:48.414454 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc65de6-9d52-4f38-a7b5-279a8499868d admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:48.418967 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599700c1-c771-4667-ab67-e3bd9a00ac2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:48.419575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599700c1-c771-4667-ab67-e3bd9a00ac2f admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:48.419862 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599700c1-c771-4667-ab67-e3bd9a00ac2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:48.420143 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599700c1-c771-4667-ab67-e3bd9a00ac2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:48.420809 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599700c1-c771-4667-ab67-e3bd9a00ac2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:48.422659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599700c1-c771-4667-ab67-e3bd9a00ac2f admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:48.473015 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 792/1573] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:23:48 2022] POST /identity/v3/users => generated 295 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:48.473015 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e587b70-8ef7-4c0b-ac9f-4abe873f4fd7 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:48.480635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e587b70-8ef7-4c0b-ac9f-4abe873f4fd7 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:48.485436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4dad4-f230-4e17-a3d4-c6ed6d60b0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:48.486083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4dad4-f230-4e17-a3d4-c6ed6d60b0d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:48.486373 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4dad4-f230-4e17-a3d4-c6ed6d60b0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:48.486658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4dad4-f230-4e17-a3d4-c6ed6d60b0d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:48.487390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4dad4-f230-4e17-a3d4-c6ed6d60b0d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:48.489291 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4dad4-f230-4e17-a3d4-c6ed6d60b0d9 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:48.619544 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 782/1574] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:23:48 2022] POST /identity/v3/projects => generated 405 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:48.620359 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599700c1-c771-4667-ab67-e3bd9a00ac2f admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:48.639727 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 793/1575] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:23:48 2022] POST /identity/v3/projects => generated 373 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:48.639727 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4dad4-f230-4e17-a3d4-c6ed6d60b0d9 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:48.661353 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:48.673288 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:48.683122 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599700c1-c771-4667-ab67-e3bd9a00ac2f admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:48.701635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4dad4-f230-4e17-a3d4-c6ed6d60b0d9 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:48.706238 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c065edb-6bf2-472a-a7e5-fd4c4b5815b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:48.706898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c065edb-6bf2-472a-a7e5-fd4c4b5815b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:48.707211 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c065edb-6bf2-472a-a7e5-fd4c4b5815b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:48.707487 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c065edb-6bf2-472a-a7e5-fd4c4b5815b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:48.708191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c065edb-6bf2-472a-a7e5-fd4c4b5815b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:48.711480 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c065edb-6bf2-472a-a7e5-fd4c4b5815b9 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:48.740624 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510a80a-f8a4-4387-bde3-f0fa2d591948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:48.741330 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510a80a-f8a4-4387-bde3-f0fa2d591948 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:48.741619 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510a80a-f8a4-4387-bde3-f0fa2d591948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:48.741900 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510a80a-f8a4-4387-bde3-f0fa2d591948 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:48.742594 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a510a80a-f8a4-4387-bde3-f0fa2d591948 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:48.744564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a510a80a-f8a4-4387-bde3-f0fa2d591948 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:48.779885 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 794/1576] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:23:48 2022] POST /identity/v3/projects => generated 365 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:48.780928 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c065edb-6bf2-472a-a7e5-fd4c4b5815b9 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:48.839661 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c065edb-6bf2-472a-a7e5-fd4c4b5815b9 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:48.844772 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7c28b-00ba-4f3c-97aa-5f277b69363a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:48.848421 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7c28b-00ba-4f3c-97aa-5f277b69363a admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:48.848421 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7c28b-00ba-4f3c-97aa-5f277b69363a admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:48.848421 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7c28b-00ba-4f3c-97aa-5f277b69363a admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:48.848421 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7c28b-00ba-4f3c-97aa-5f277b69363a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:48.848421 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7c28b-00ba-4f3c-97aa-5f277b69363a admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:48.858699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 795/1577] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:48.859637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7c28b-00ba-4f3c-97aa-5f277b69363a admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:48.868716 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7c28b-00ba-4f3c-97aa-5f277b69363a admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:48.873114 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3278cd6-a193-4224-adb0-b10827880be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:48.873749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3278cd6-a193-4224-adb0-b10827880be1 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:48.874035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3278cd6-a193-4224-adb0-b10827880be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:48.874310 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3278cd6-a193-4224-adb0-b10827880be1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:48.875008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3278cd6-a193-4224-adb0-b10827880be1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:48.876860 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3278cd6-a193-4224-adb0-b10827880be1 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:48.893635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 783/1578] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:23:48 2022] POST /identity/v3/projects => generated 391 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:48.895521 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510a80a-f8a4-4387-bde3-f0fa2d591948 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:48.961236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510a80a-f8a4-4387-bde3-f0fa2d591948 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:48.966425 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26283219-e15a-4d69-84d2-a2da983f09e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:48.967116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26283219-e15a-4d69-84d2-a2da983f09e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:48.967414 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26283219-e15a-4d69-84d2-a2da983f09e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:48.967696 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26283219-e15a-4d69-84d2-a2da983f09e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:48.968384 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26283219-e15a-4d69-84d2-a2da983f09e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:48.970432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26283219-e15a-4d69-84d2-a2da983f09e3 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:48.996092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 796/1579] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:23:48 2022] POST /identity/v3/projects => generated 379 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:48.997063 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3278cd6-a193-4224-adb0-b10827880be1 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:49.057584 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3278cd6-a193-4224-adb0-b10827880be1 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:49.062426 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed5e2c-3b8f-426c-80b5-b1fae3640872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:49.063128 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed5e2c-3b8f-426c-80b5-b1fae3640872 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:49.063417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed5e2c-3b8f-426c-80b5-b1fae3640872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:49.063701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed5e2c-3b8f-426c-80b5-b1fae3640872 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:49.064429 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed5e2c-3b8f-426c-80b5-b1fae3640872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:49.066030 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed5e2c-3b8f-426c-80b5-b1fae3640872 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:49.074981 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 784/1580] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:23:48 2022] POST /identity/v3/projects => generated 373 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:49.077011 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26283219-e15a-4d69-84d2-a2da983f09e3 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:49.077152 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 797/1581] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:49.077497 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed5e2c-3b8f-426c-80b5-b1fae3640872 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:49.087635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed5e2c-3b8f-426c-80b5-b1fae3640872 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:49.099679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:49.145553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c11b1-c077-43ce-9998-988f0e2e7de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:49.145860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26283219-e15a-4d69-84d2-a2da983f09e3 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:49.146152 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c11b1-c077-43ce-9998-988f0e2e7de4 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:49.146442 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c11b1-c077-43ce-9998-988f0e2e7de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:49.146724 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c11b1-c077-43ce-9998-988f0e2e7de4 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:49.147469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c11b1-c077-43ce-9998-988f0e2e7de4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:49.149486 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c11b1-c077-43ce-9998-988f0e2e7de4 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:49.154532 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5da4e-2a55-4b5f-95c3-8f57e59b2b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:49.155292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5da4e-2a55-4b5f-95c3-8f57e59b2b2a admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:49.155574 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5da4e-2a55-4b5f-95c3-8f57e59b2b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:49.155859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5da4e-2a55-4b5f-95c3-8f57e59b2b2a admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:49.156592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5da4e-2a55-4b5f-95c3-8f57e59b2b2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:49.158528 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5da4e-2a55-4b5f-95c3-8f57e59b2b2a admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:49.287704 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:49.292305 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 798/1582] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:23:49 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:49.292909 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 785/1583] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:23:49 2022] POST /identity/v3/users => generated 303 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:49.293144 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c11b1-c077-43ce-9998-988f0e2e7de4 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:49.293881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5da4e-2a55-4b5f-95c3-8f57e59b2b2a admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:49.303853 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c11b1-c077-43ce-9998-988f0e2e7de4 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:49.304066 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5da4e-2a55-4b5f-95c3-8f57e59b2b2a admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:49.308437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d5a60-b295-460e-94e0-fd3248a4663c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:49.309098 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d5a60-b295-460e-94e0-fd3248a4663c admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:49.309254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162c0f2-1d2e-411b-98c5-82946f8da3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:49.309387 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d5a60-b295-460e-94e0-fd3248a4663c admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:49.309645 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d5a60-b295-460e-94e0-fd3248a4663c admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:49.310359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162c0f2-1d2e-411b-98c5-82946f8da3e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:49.310480 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d5a60-b295-460e-94e0-fd3248a4663c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:49.310659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162c0f2-1d2e-411b-98c5-82946f8da3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:49.310983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162c0f2-1d2e-411b-98c5-82946f8da3e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:49.311818 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162c0f2-1d2e-411b-98c5-82946f8da3e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:49.312006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d5a60-b295-460e-94e0-fd3248a4663c admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:49.313787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162c0f2-1d2e-411b-98c5-82946f8da3e6 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:49.322017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 786/1584] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:49.323127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d5a60-b295-460e-94e0-fd3248a4663c admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:49.333355 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d5a60-b295-460e-94e0-fd3248a4663c admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:49.337875 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc5a83-ff92-4c64-b095-de2dd45f27f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:49.338504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc5a83-ff92-4c64-b095-de2dd45f27f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:49.338788 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc5a83-ff92-4c64-b095-de2dd45f27f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:49.339108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc5a83-ff92-4c64-b095-de2dd45f27f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:49.339785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc5a83-ff92-4c64-b095-de2dd45f27f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:49.342273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc5a83-ff92-4c64-b095-de2dd45f27f5 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:49.412239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 799/1585] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:23:49 2022] POST /identity/v3/users => generated 283 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:49.412880 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162c0f2-1d2e-411b-98c5-82946f8da3e6 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:49.422051 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162c0f2-1d2e-411b-98c5-82946f8da3e6 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:49.426584 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9487d4-a23c-4d95-aace-23de490bcfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:49.427250 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9487d4-a23c-4d95-aace-23de490bcfeb admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:49.427536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9487d4-a23c-4d95-aace-23de490bcfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:49.427813 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9487d4-a23c-4d95-aace-23de490bcfeb admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:49.428491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9487d4-a23c-4d95-aace-23de490bcfeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:49.430383 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9487d4-a23c-4d95-aace-23de490bcfeb admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:49.458195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 787/1586] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:23:49 2022] POST /identity/v3/users => generated 296 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:49.460739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc5a83-ff92-4c64-b095-de2dd45f27f5 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:49.468225 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc5a83-ff92-4c64-b095-de2dd45f27f5 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:49.472629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d33eff-bde8-41cc-a898-a442f79b16ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:49.473253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d33eff-bde8-41cc-a898-a442f79b16ba admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:49.473538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d33eff-bde8-41cc-a898-a442f79b16ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:49.473813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d33eff-bde8-41cc-a898-a442f79b16ba admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:49.474480 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d33eff-bde8-41cc-a898-a442f79b16ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:49.476424 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d33eff-bde8-41cc-a898-a442f79b16ba admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:49.618638 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 800/1587] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:23:49 2022] POST /identity/v3/users => generated 290 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:49.619623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9487d4-a23c-4d95-aace-23de490bcfeb admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:49.629717 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9487d4-a23c-4d95-aace-23de490bcfeb admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:49.634810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb6857-cbc8-4e7b-8378-a5e280808f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:49.635854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb6857-cbc8-4e7b-8378-a5e280808f39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:49.636164 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb6857-cbc8-4e7b-8378-a5e280808f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:49.636441 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb6857-cbc8-4e7b-8378-a5e280808f39 admin admin] PATH_INFO: `/v3/projects/e69f07e6433b48a28cb628a84639a85c/users/aa441aefdec64a74ab2f0aa1554433a9/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:49.642618 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 788/1588] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:23:49 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:49.643465 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d33eff-bde8-41cc-a898-a442f79b16ba admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:49.652426 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb6857-cbc8-4e7b-8378-a5e280808f39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e69f07e6433b48a28cb628a84639a85c, user_id=aa441aefdec64a74ab2f0aa1554433a9, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:49.653282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d33eff-bde8-41cc-a898-a442f79b16ba admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:49.655095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb6857-cbc8-4e7b-8378-a5e280808f39 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:49.657944 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7ec44-2be2-495a-ae53-edb845284284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:49.658597 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7ec44-2be2-495a-ae53-edb845284284 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:49.658886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7ec44-2be2-495a-ae53-edb845284284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:49.659201 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7ec44-2be2-495a-ae53-edb845284284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:49.659922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7ec44-2be2-495a-ae53-edb845284284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:49.661531 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7ec44-2be2-495a-ae53-edb845284284 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:49.671438 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 789/1589] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:49.672286 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7ec44-2be2-495a-ae53-edb845284284 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:49.681379 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7ec44-2be2-495a-ae53-edb845284284 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:49.686309 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507ea88-b81d-4585-be75-c3ffd679dec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:49.686961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507ea88-b81d-4585-be75-c3ffd679dec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:49.687257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507ea88-b81d-4585-be75-c3ffd679dec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:49.687536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507ea88-b81d-4585-be75-c3ffd679dec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:49.688224 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507ea88-b81d-4585-be75-c3ffd679dec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:49.689754 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507ea88-b81d-4585-be75-c3ffd679dec8 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:49.699148 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 790/1590] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:49.700133 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507ea88-b81d-4585-be75-c3ffd679dec8 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:49.709842 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507ea88-b81d-4585-be75-c3ffd679dec8 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:49.714310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c11e6-1a8e-4f8b-88fc-32fbcb425246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:49.715177 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c11e6-1a8e-4f8b-88fc-32fbcb425246 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:49.715469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c11e6-1a8e-4f8b-88fc-32fbcb425246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:49.715749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c11e6-1a8e-4f8b-88fc-32fbcb425246 admin admin] PATH_INFO: `/v3/projects/29ece34041e04c78868e53d90b658a82/users/30bc3dba44ed4d14a67e56aa640bf9ca/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:49.729935 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c11e6-1a8e-4f8b-88fc-32fbcb425246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ece34041e04c78868e53d90b658a82, user_id=30bc3dba44ed4d14a67e56aa640bf9ca, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:49.730959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: WARNING py.warnings [None req-fb8c11e6-1a8e-4f8b-88fc-32fbcb425246 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 09:23:49.730959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: warnings.warn(msg) Jun 23 09:23:49.730959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: Jun 23 09:23:49.733167 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c11e6-1a8e-4f8b-88fc-32fbcb425246 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:49.777453 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 801/1591] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:23:49 2022] PUT /identity/v3/projects/e69f07e6433b48a28cb628a84639a85c/users/aa441aefdec64a74ab2f0aa1554433a9/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:23:49.778277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb6857-cbc8-4e7b-8378-a5e280808f39 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:49.801298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 791/1592] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:23:49 2022] PUT /identity/v3/projects/29ece34041e04c78868e53d90b658a82/users/30bc3dba44ed4d14a67e56aa640bf9ca/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:23:49.802170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c11e6-1a8e-4f8b-88fc-32fbcb425246 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:49.839637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb6857-cbc8-4e7b-8378-a5e280808f39 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:49.863291 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c11e6-1a8e-4f8b-88fc-32fbcb425246 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:49.867797 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55857940-73e0-483b-8c7d-d1d6e3682506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:49.868459 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55857940-73e0-483b-8c7d-d1d6e3682506 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:49.868743 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55857940-73e0-483b-8c7d-d1d6e3682506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:49.869018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55857940-73e0-483b-8c7d-d1d6e3682506 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:49.869742 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55857940-73e0-483b-8c7d-d1d6e3682506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:49.871385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55857940-73e0-483b-8c7d-d1d6e3682506 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:49.881565 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 792/1593] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:49.882355 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55857940-73e0-483b-8c7d-d1d6e3682506 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:49.890259 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:49.891648 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55857940-73e0-483b-8c7d-d1d6e3682506 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:49.896189 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf999c6-d28f-4617-8fcd-d613388fb072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:49.899228 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf999c6-d28f-4617-8fcd-d613388fb072 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:49.899228 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf999c6-d28f-4617-8fcd-d613388fb072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:49.899228 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf999c6-d28f-4617-8fcd-d613388fb072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:49.899228 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf999c6-d28f-4617-8fcd-d613388fb072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:49.899647 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf999c6-d28f-4617-8fcd-d613388fb072 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:49.899756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb284bd-ada3-4887-8978-e489082fb4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:49.900365 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb284bd-ada3-4887-8978-e489082fb4f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:49.900659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb284bd-ada3-4887-8978-e489082fb4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:49.900942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb284bd-ada3-4887-8978-e489082fb4f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:49.901805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb284bd-ada3-4887-8978-e489082fb4f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:49.903449 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb284bd-ada3-4887-8978-e489082fb4f7 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:49.912865 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 793/1594] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:49.912865 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf999c6-d28f-4617-8fcd-d613388fb072 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:49.914163 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 802/1595] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:49.915627 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb284bd-ada3-4887-8978-e489082fb4f7 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:49.925321 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf999c6-d28f-4617-8fcd-d613388fb072 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:49.927830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb284bd-ada3-4887-8978-e489082fb4f7 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:49.928742 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2c017-ff42-479e-a35f-ca87a45f0f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:49.929387 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2c017-ff42-479e-a35f-ca87a45f0f65 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:49.929672 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2c017-ff42-479e-a35f-ca87a45f0f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:49.929948 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2c017-ff42-479e-a35f-ca87a45f0f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:49.930670 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2c017-ff42-479e-a35f-ca87a45f0f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:49.932314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2c017-ff42-479e-a35f-ca87a45f0f65 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:49.932565 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc978cc-7a9f-441e-b946-d7c447292e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:49.933227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc978cc-7a9f-441e-b946-d7c447292e0f admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:49.933518 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc978cc-7a9f-441e-b946-d7c447292e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:49.933799 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc978cc-7a9f-441e-b946-d7c447292e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:49.934532 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc978cc-7a9f-441e-b946-d7c447292e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:49.936294 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc978cc-7a9f-441e-b946-d7c447292e0f admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:49.943828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 794/1596] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:49.944467 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2c017-ff42-479e-a35f-ca87a45f0f65 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:49.949676 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 803/1597] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:49.949676 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc978cc-7a9f-441e-b946-d7c447292e0f admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:49.954577 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2c017-ff42-479e-a35f-ca87a45f0f65 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:49.959201 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5a617-a8fc-49ab-bb98-9d666b8eab4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:49.959850 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5a617-a8fc-49ab-bb98-9d666b8eab4a admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:49.960134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5a617-a8fc-49ab-bb98-9d666b8eab4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:49.960242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc978cc-7a9f-441e-b946-d7c447292e0f admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:49.961009 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5a617-a8fc-49ab-bb98-9d666b8eab4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:49.961111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5a617-a8fc-49ab-bb98-9d666b8eab4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:49.962694 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5a617-a8fc-49ab-bb98-9d666b8eab4a admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:49.964913 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89944b43-7c82-4faa-abce-e8828c2e89f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:49.965563 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89944b43-7c82-4faa-abce-e8828c2e89f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:49.965880 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89944b43-7c82-4faa-abce-e8828c2e89f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:49.966195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89944b43-7c82-4faa-abce-e8828c2e89f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:49.966982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89944b43-7c82-4faa-abce-e8828c2e89f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:49.968567 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89944b43-7c82-4faa-abce-e8828c2e89f3 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:49.972666 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 795/1598] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:49.973856 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b909a-343f-4738-99bc-df116dfc35c8 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:49.974142 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b909a-343f-4738-99bc-df116dfc35c8 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:49.974420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b909a-343f-4738-99bc-df116dfc35c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:49.979439 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 804/1599] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:49.980518 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89944b43-7c82-4faa-abce-e8828c2e89f3 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:49.991238 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89944b43-7c82-4faa-abce-e8828c2e89f3 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:49.992086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-a81b909a-343f-4738-99bc-df116dfc35c8 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:49.993285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a81b909a-343f-4738-99bc-df116dfc35c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:49.995858 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119e347-5455-4664-8799-33ad75373cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:49.996504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119e347-5455-4664-8799-33ad75373cec admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:49.996827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119e347-5455-4664-8799-33ad75373cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:49.997109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119e347-5455-4664-8799-33ad75373cec admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:49.997898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119e347-5455-4664-8799-33ad75373cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:49.999671 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119e347-5455-4664-8799-33ad75373cec admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:50.000884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 796/1600] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:50.001614 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b909a-343f-4738-99bc-df116dfc35c8 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:50.013894 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 805/1601] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:50.013894 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119e347-5455-4664-8799-33ad75373cec admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:50.016790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b909a-343f-4738-99bc-df116dfc35c8 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:50.021172 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119e347-5455-4664-8799-33ad75373cec admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:50.021372 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7134f4-870d-40a2-8282-062ed2989abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:50.022029 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7134f4-870d-40a2-8282-062ed2989abf admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:50.022320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7134f4-870d-40a2-8282-062ed2989abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:50.022643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7134f4-870d-40a2-8282-062ed2989abf admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:50.023362 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7134f4-870d-40a2-8282-062ed2989abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:50.024962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7134f4-870d-40a2-8282-062ed2989abf admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:50.025884 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0c016-988d-42b9-9efd-f855143e0c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:50.026531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0c016-988d-42b9-9efd-f855143e0c89 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:50.026819 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0c016-988d-42b9-9efd-f855143e0c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:50.027136 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0c016-988d-42b9-9efd-f855143e0c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:50.027869 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0c016-988d-42b9-9efd-f855143e0c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:50.029486 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0c016-988d-42b9-9efd-f855143e0c89 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:50.035261 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 797/1602] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:50.036044 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7134f4-870d-40a2-8282-062ed2989abf admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:50.043523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 806/1603] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:50.043523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0c016-988d-42b9-9efd-f855143e0c89 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:50.046796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7134f4-870d-40a2-8282-062ed2989abf admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:50.053754 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9ac2f-957d-4e2c-9817-417cc46f1563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:50.054430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9ac2f-957d-4e2c-9817-417cc46f1563 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:50.054715 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9ac2f-957d-4e2c-9817-417cc46f1563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:50.055025 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9ac2f-957d-4e2c-9817-417cc46f1563 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:50.055767 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d9ac2f-957d-4e2c-9817-417cc46f1563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:50.057379 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d9ac2f-957d-4e2c-9817-417cc46f1563 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:50.057570 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0c016-988d-42b9-9efd-f855143e0c89 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:50.062381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149606e-c997-4353-b7a2-022926ecd700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:50.063297 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149606e-c997-4353-b7a2-022926ecd700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:50.063582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149606e-c997-4353-b7a2-022926ecd700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:50.063860 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149606e-c997-4353-b7a2-022926ecd700 admin admin] PATH_INFO: `/v3/projects/ad2e6e427876444390f75d4cc8080347/users/4570d1fead8c4328a99cd58a02867c45/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:50.067948 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 798/1604] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:50.068962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9ac2f-957d-4e2c-9817-417cc46f1563 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:50.074124 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:50.079447 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149606e-c997-4353-b7a2-022926ecd700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad2e6e427876444390f75d4cc8080347, user_id=4570d1fead8c4328a99cd58a02867c45, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:50.079821 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9ac2f-957d-4e2c-9817-417cc46f1563 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:50.082152 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149606e-c997-4353-b7a2-022926ecd700 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:50.084582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965edc24-481f-4015-804e-9878573c0ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:50.085447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965edc24-481f-4015-804e-9878573c0ac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:50.085731 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965edc24-481f-4015-804e-9878573c0ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:50.086146 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965edc24-481f-4015-804e-9878573c0ac3 admin admin] PATH_INFO: `/v3/projects/75ca8c6ca93741a085c55058a543b245/users/bc50bc409c7e401f9294202d641db3cf/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:50.100294 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965edc24-481f-4015-804e-9878573c0ac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75ca8c6ca93741a085c55058a543b245, user_id=bc50bc409c7e401f9294202d641db3cf, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:50.103046 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965edc24-481f-4015-804e-9878573c0ac3 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:50.148894 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 807/1605] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:23:50 2022] PUT /identity/v3/projects/ad2e6e427876444390f75d4cc8080347/users/4570d1fead8c4328a99cd58a02867c45/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:23:50.149699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149606e-c997-4353-b7a2-022926ecd700 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:50.172912 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 799/1606] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:23:50 2022] PUT /identity/v3/projects/75ca8c6ca93741a085c55058a543b245/users/bc50bc409c7e401f9294202d641db3cf/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:23:50.173629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965edc24-481f-4015-804e-9878573c0ac3 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:50.226720 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149606e-c997-4353-b7a2-022926ecd700 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:50.227920 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965edc24-481f-4015-804e-9878573c0ac3 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:50.232332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a69c6-26b5-4db8-88d0-8b312195e61c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:50.232979 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a69c6-26b5-4db8-88d0-8b312195e61c service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:50.233262 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a69c6-26b5-4db8-88d0-8b312195e61c service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:50.233548 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a69c6-26b5-4db8-88d0-8b312195e61c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:50.283185 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042bb12-5c08-4507-822f-dc318f8cff8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:50.284082 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042bb12-5c08-4507-822f-dc318f8cff8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:50.284366 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042bb12-5c08-4507-822f-dc318f8cff8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:50.284641 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042bb12-5c08-4507-822f-dc318f8cff8f admin admin] PATH_INFO: `/v3/projects/e69f07e6433b48a28cb628a84639a85c/users/aa441aefdec64a74ab2f0aa1554433a9/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:50.286612 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042bb12-5c08-4507-822f-dc318f8cff8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e69f07e6433b48a28cb628a84639a85c, user_id=aa441aefdec64a74ab2f0aa1554433a9, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:50.289332 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042bb12-5c08-4507-822f-dc318f8cff8f admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:50.293132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a69c6-26b5-4db8-88d0-8b312195e61c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:50.294650 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a69c6-26b5-4db8-88d0-8b312195e61c service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:50.300038 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 808/1607] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:23:50 2022] PUT /identity/v3/projects/e69f07e6433b48a28cb628a84639a85c/users/aa441aefdec64a74ab2f0aa1554433a9/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:23:50.300949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042bb12-5c08-4507-822f-dc318f8cff8f admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:50.305093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 800/1608] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:23:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:23:50.306125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a69c6-26b5-4db8-88d0-8b312195e61c service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:50.316376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a69c6-26b5-4db8-88d0-8b312195e61c service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:50.333785 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:50.369795 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042bb12-5c08-4507-822f-dc318f8cff8f admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:50.373503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465021a-444a-431f-b727-a37973303146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:50.374393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465021a-444a-431f-b727-a37973303146 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:50.374798 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465021a-444a-431f-b727-a37973303146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:50.375022 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0ce02-8e1a-4fc6-92ab-0773cb7b616e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:50.375154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465021a-444a-431f-b727-a37973303146 admin admin] PATH_INFO: `/v3/projects/0160aecb23854482b26d188beec1021d/users/6e7f2d909d45471fb0b97ddb724cb195/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:50.375938 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0ce02-8e1a-4fc6-92ab-0773cb7b616e admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:50.376224 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0ce02-8e1a-4fc6-92ab-0773cb7b616e admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:50.376567 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0ce02-8e1a-4fc6-92ab-0773cb7b616e admin admin] PATH_INFO: `/v3/projects/368a6f8fbca64e7791d5330f0a99d086/users/29fe5fb181e2452388d24fd0b7aa5e9f/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:50.390206 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465021a-444a-431f-b727-a37973303146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0160aecb23854482b26d188beec1021d, user_id=6e7f2d909d45471fb0b97ddb724cb195, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:50.390852 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d0ce02-8e1a-4fc6-92ab-0773cb7b616e admin admin] RBAC: Authorizing `identity:create_grant(project_id=368a6f8fbca64e7791d5330f0a99d086, user_id=29fe5fb181e2452388d24fd0b7aa5e9f, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:50.392945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465021a-444a-431f-b727-a37973303146 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:50.393537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d0ce02-8e1a-4fc6-92ab-0773cb7b616e admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:50.481221 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 801/1609] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:23:50 2022] PUT /identity/v3/projects/0160aecb23854482b26d188beec1021d/users/6e7f2d909d45471fb0b97ddb724cb195/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:23:50.482613 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465021a-444a-431f-b727-a37973303146 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:50.495734 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:50.509950 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 809/1610] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:23:50 2022] PUT /identity/v3/projects/368a6f8fbca64e7791d5330f0a99d086/users/29fe5fb181e2452388d24fd0b7aa5e9f/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:23:50.511057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0ce02-8e1a-4fc6-92ab-0773cb7b616e admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:50.548005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465021a-444a-431f-b727-a37973303146 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:50.574370 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0ce02-8e1a-4fc6-92ab-0773cb7b616e admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:50.580319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48637510-d4b1-4377-addf-62c90b8da750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:50.581239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48637510-d4b1-4377-addf-62c90b8da750 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:50.581524 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48637510-d4b1-4377-addf-62c90b8da750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:50.581802 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48637510-d4b1-4377-addf-62c90b8da750 admin admin] PATH_INFO: `/v3/projects/79e51d48f9dc4c04b00dc0b5da9ded58/users/911ff3294dfe46509f74c3cb8c558863/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:50.596399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48637510-d4b1-4377-addf-62c90b8da750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79e51d48f9dc4c04b00dc0b5da9ded58, user_id=911ff3294dfe46509f74c3cb8c558863, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:50.599175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48637510-d4b1-4377-addf-62c90b8da750 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:50.603227 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c086f66-526a-4201-a275-5efa57fff967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:50.604113 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c086f66-526a-4201-a275-5efa57fff967 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:50.604400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c086f66-526a-4201-a275-5efa57fff967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:50.604900 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c086f66-526a-4201-a275-5efa57fff967 admin admin] PATH_INFO: `/v3/projects/7e77ff181ca6451d9039171e88e31f5f/users/58ec7836481d4b4ab65985d52e03d33f/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:50.620351 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c086f66-526a-4201-a275-5efa57fff967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e77ff181ca6451d9039171e88e31f5f, user_id=58ec7836481d4b4ab65985d52e03d33f, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:50.623100 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c086f66-526a-4201-a275-5efa57fff967 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:50.674678 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 810/1611] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:23:50 2022] PUT /identity/v3/projects/79e51d48f9dc4c04b00dc0b5da9ded58/users/911ff3294dfe46509f74c3cb8c558863/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:23:50.685633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b54bfd-8476-4247-8a7c-db1999d860ac None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:50.685933 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b54bfd-8476-4247-8a7c-db1999d860ac None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:50.686216 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b54bfd-8476-4247-8a7c-db1999d860ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:50.701016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 802/1612] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:23:50 2022] PUT /identity/v3/projects/7e77ff181ca6451d9039171e88e31f5f/users/58ec7836481d4b4ab65985d52e03d33f/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:23:50.702878 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b703472-998e-49b7-85b0-160148d5362c None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:50.703344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b703472-998e-49b7-85b0-160148d5362c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:50.703777 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b703472-998e-49b7-85b0-160148d5362c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:50.723518 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-87b54bfd-8476-4247-8a7c-db1999d860ac None None] MFA Rules not processed for user `4570d1fead8c4328a99cd58a02867c45`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:50.725221 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-87b54bfd-8476-4247-8a7c-db1999d860ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:50.738109 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-8b703472-998e-49b7-85b0-160148d5362c None None] MFA Rules not processed for user `bc50bc409c7e401f9294202d641db3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:50.739279 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-8b703472-998e-49b7-85b0-160148d5362c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:50.791917 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 811/1613] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:50.793222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5b2f9-5cef-4cde-840e-df86ad9b6da1 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:50.794620 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5b2f9-5cef-4cde-840e-df86ad9b6da1 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:50.803883 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 803/1614] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:50 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:50.805548 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4aae7f-cceb-4a79-b677-bee113a3c0dc None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:50.805682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4aae7f-cceb-4a79-b677-bee113a3c0dc None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:50.805877 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4aae7f-cceb-4a79-b677-bee113a3c0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:50.807854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5b2f9-5cef-4cde-840e-df86ad9b6da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:50.827239 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-4d4aae7f-cceb-4a79-b677-bee113a3c0dc None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:50.828512 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-4d4aae7f-cceb-4a79-b677-bee113a3c0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:50.842536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-62f5b2f9-5cef-4cde-840e-df86ad9b6da1 None None] MFA Rules not processed for user `aa441aefdec64a74ab2f0aa1554433a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:50.843786 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-62f5b2f9-5cef-4cde-840e-df86ad9b6da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:50.849342 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:50.859951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:50.901432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 804/1615] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:50.903961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f4728-0bab-4de7-862e-4529bae252a5 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:50.904435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f4728-0bab-4de7-862e-4529bae252a5 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:50.905221 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f4728-0bab-4de7-862e-4529bae252a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:50.930752 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-200f4728-0bab-4de7-862e-4529bae252a5 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:50.931738 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 812/1616] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:50.932195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-200f4728-0bab-4de7-862e-4529bae252a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:50.933190 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d03a1-5169-4577-af45-c467a27a7dba None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:50.933541 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d03a1-5169-4577-af45-c467a27a7dba None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:50.934023 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d03a1-5169-4577-af45-c467a27a7dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:50.945528 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 805/1617] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:50.947373 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9e1de-278f-4cdd-bbfe-e370c07121e1 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:50.947876 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9e1de-278f-4cdd-bbfe-e370c07121e1 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:50.954758 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9e1de-278f-4cdd-bbfe-e370c07121e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:50.967006 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-de1d03a1-5169-4577-af45-c467a27a7dba None None] MFA Rules not processed for user `911ff3294dfe46509f74c3cb8c558863`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:50.968320 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-de1d03a1-5169-4577-af45-c467a27a7dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:50.986165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-8ee9e1de-278f-4cdd-bbfe-e370c07121e1 None None] MFA Rules not processed for user `58ec7836481d4b4ab65985d52e03d33f`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:50.987464 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-8ee9e1de-278f-4cdd-bbfe-e370c07121e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:51.046457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 813/1618] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:51.046457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d03a1-5169-4577-af45-c467a27a7dba None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:51.055306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d03a1-5169-4577-af45-c467a27a7dba None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:51.063977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 806/1619] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:50 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:51.064152 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f6d32-6818-40a8-ac88-eec630bc2a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:51.064152 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f6d32-6818-40a8-ac88-eec630bc2a50 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:51.064596 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6215665-ab9f-464a-866c-a7b4f4940bdb None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:51.064719 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f6d32-6818-40a8-ac88-eec630bc2a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:51.064823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6215665-ab9f-464a-866c-a7b4f4940bdb None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:51.064916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f6d32-6818-40a8-ac88-eec630bc2a50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:51.065022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6215665-ab9f-464a-866c-a7b4f4940bdb None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:51.065114 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f6d32-6818-40a8-ac88-eec630bc2a50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:51.065205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 807/1620] 10.222.0.52 () {54 vars in 961 bytes} [Thu Jun 23 09:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:23:51.065205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6215665-ab9f-464a-866c-a7b4f4940bdb None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:51.065736 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f6d32-6818-40a8-ac88-eec630bc2a50 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:51.066754 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a6215665-ab9f-464a-866c-a7b4f4940bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:51.126588 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6215665-ab9f-464a-866c-a7b4f4940bdb None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:51.130170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03cd4b-efd7-478d-88e2-a0d2d7e9fe61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:51.130823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03cd4b-efd7-478d-88e2-a0d2d7e9fe61 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:51.131131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03cd4b-efd7-478d-88e2-a0d2d7e9fe61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:51.131410 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03cd4b-efd7-478d-88e2-a0d2d7e9fe61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:51.140874 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e03cd4b-efd7-478d-88e2-a0d2d7e9fe61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:51.142310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e03cd4b-efd7-478d-88e2-a0d2d7e9fe61 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:51.152256 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 808/1621] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:23:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:23:51.153098 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03cd4b-efd7-478d-88e2-a0d2d7e9fe61 service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:51.178900 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 814/1622] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:23:51 2022] POST /identity/v3/projects => generated 405 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:51.179993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f6d32-6818-40a8-ac88-eec630bc2a50 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:51.216171 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03cd4b-efd7-478d-88e2-a0d2d7e9fe61 service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:51.244210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f6d32-6818-40a8-ac88-eec630bc2a50 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:51.249356 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d422a4-e772-4e78-9a13-3331951aa3b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:51.250019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d422a4-e772-4e78-9a13-3331951aa3b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:51.250783 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d422a4-e772-4e78-9a13-3331951aa3b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:51.251052 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d422a4-e772-4e78-9a13-3331951aa3b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:51.276897 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d3a06-f8df-4ede-8d59-70b851b7f1b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:51.277547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d3a06-f8df-4ede-8d59-70b851b7f1b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:51.277828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d3a06-f8df-4ede-8d59-70b851b7f1b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:51.278105 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d3a06-f8df-4ede-8d59-70b851b7f1b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:51.314218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d422a4-e772-4e78-9a13-3331951aa3b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:51.315878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d422a4-e772-4e78-9a13-3331951aa3b9 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:51.328503 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 815/1623] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:23:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:23:51.329471 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d422a4-e772-4e78-9a13-3331951aa3b9 service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:51.338857 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d422a4-e772-4e78-9a13-3331951aa3b9 service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:51.344972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf0232-1999-461f-9739-3e4fcb12c7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:51.345655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf0232-1999-461f-9739-3e4fcb12c7e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:51.345943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf0232-1999-461f-9739-3e4fcb12c7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:51.346540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf0232-1999-461f-9739-3e4fcb12c7e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:51.347303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf0232-1999-461f-9739-3e4fcb12c7e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:51.349764 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf0232-1999-461f-9739-3e4fcb12c7e5 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:51.354102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d3a06-f8df-4ede-8d59-70b851b7f1b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:51.354462 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:51.356016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d3a06-f8df-4ede-8d59-70b851b7f1b2 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:51.368469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 809/1624] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:23:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:23:51.369530 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d3a06-f8df-4ede-8d59-70b851b7f1b2 service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:51.380792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d3a06-f8df-4ede-8d59-70b851b7f1b2 service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:51.385592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ffe3f-5878-4877-a481-590d06532112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:51.386263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ffe3f-5878-4877-a481-590d06532112 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:51.386556 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ffe3f-5878-4877-a481-590d06532112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:51.386845 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ffe3f-5878-4877-a481-590d06532112 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:51.388691 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ffe3f-5878-4877-a481-590d06532112 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:51.390700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ffe3f-5878-4877-a481-590d06532112 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:51.408125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:51.476540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 816/1625] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:23:51 2022] POST /identity/v3/projects => generated 371 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:51.476540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e14030-4a93-4e58-898f-c8d5376e4eb9 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:51.476540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e14030-4a93-4e58-898f-c8d5376e4eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:51.476540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e14030-4a93-4e58-898f-c8d5376e4eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:51.501267 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-a5e14030-4a93-4e58-898f-c8d5376e4eb9 None None] MFA Rules not processed for user `ff025cb706d243778468ae4570374690`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:51.505196 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 810/1626] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:23:51 2022] POST /identity/v3/projects => generated 365 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:51.505316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-a5e14030-4a93-4e58-898f-c8d5376e4eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:51.505421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ffe3f-5878-4877-a481-590d06532112 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:51.594662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ffe3f-5878-4877-a481-590d06532112 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:51.605269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74248406-b100-4fdc-bf19-cd2743910b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:51.606236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74248406-b100-4fdc-bf19-cd2743910b53 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:51.607909 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74248406-b100-4fdc-bf19-cd2743910b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:51.608472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74248406-b100-4fdc-bf19-cd2743910b53 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:51.609604 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74248406-b100-4fdc-bf19-cd2743910b53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:51.613249 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74248406-b100-4fdc-bf19-cd2743910b53 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:51.621600 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 817/1627] 10.222.0.52 () {58 vars in 1058 bytes} [Thu Jun 23 09:23:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:51.623516 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e14030-4a93-4e58-898f-c8d5376e4eb9 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:51.635313 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e14030-4a93-4e58-898f-c8d5376e4eb9 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:51.640921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9216ae-2faf-4fba-969f-fcae7ed43347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:51.641612 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9216ae-2faf-4fba-969f-fcae7ed43347 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:51.641906 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9216ae-2faf-4fba-969f-fcae7ed43347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:51.642189 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9216ae-2faf-4fba-969f-fcae7ed43347 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:51.642889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9216ae-2faf-4fba-969f-fcae7ed43347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:51.644954 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9216ae-2faf-4fba-969f-fcae7ed43347 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:51.742658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 811/1628] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:23:51 2022] POST /identity/v3/users => generated 309 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:51.742658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74248406-b100-4fdc-bf19-cd2743910b53 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:51.743840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 818/1629] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:23:51 2022] POST /identity/v3/users => generated 292 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:51.744680 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9216ae-2faf-4fba-969f-fcae7ed43347 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:51.752839 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74248406-b100-4fdc-bf19-cd2743910b53 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:51.758144 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9216ae-2faf-4fba-969f-fcae7ed43347 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:51.758598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9daa6c-1a36-4763-a589-da1eff62e344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:51.759537 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9daa6c-1a36-4763-a589-da1eff62e344 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:51.759827 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9daa6c-1a36-4763-a589-da1eff62e344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:51.760106 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9daa6c-1a36-4763-a589-da1eff62e344 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:51.761006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9daa6c-1a36-4763-a589-da1eff62e344 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:51.763064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9daa6c-1a36-4763-a589-da1eff62e344 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:51.764860 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b41538-6840-475c-be7f-dbf09bd6f0f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:51.765629 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b41538-6840-475c-be7f-dbf09bd6f0f1 service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:51.766020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b41538-6840-475c-be7f-dbf09bd6f0f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:51.766375 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b41538-6840-475c-be7f-dbf09bd6f0f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:51.827409 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b41538-6840-475c-be7f-dbf09bd6f0f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:51.829099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b41538-6840-475c-be7f-dbf09bd6f0f1 service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:51.838712 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 819/1630] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:23:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:23:51.839595 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b41538-6840-475c-be7f-dbf09bd6f0f1 service nova] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:51.852468 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b41538-6840-475c-be7f-dbf09bd6f0f1 service nova] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:51.864370 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff59c3-bcf5-4188-830e-b1ea15bab812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:51.865347 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff59c3-bcf5-4188-830e-b1ea15bab812 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:51.865786 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff59c3-bcf5-4188-830e-b1ea15bab812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:51.866240 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff59c3-bcf5-4188-830e-b1ea15bab812 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:51.867340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff59c3-bcf5-4188-830e-b1ea15bab812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:51.871226 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff59c3-bcf5-4188-830e-b1ea15bab812 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:51.880246 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:51.889899 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 820/1631] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:51.889899 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff59c3-bcf5-4188-830e-b1ea15bab812 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:51.900778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 812/1632] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:23:51 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:23:51.904197 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9daa6c-1a36-4763-a589-da1eff62e344 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:51.906553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff59c3-bcf5-4188-830e-b1ea15bab812 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:51.912309 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2272d-300e-4106-a9d7-58b7158aaf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:51.913216 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2272d-300e-4106-a9d7-58b7158aaf32 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:51.913634 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2272d-300e-4106-a9d7-58b7158aaf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:51.913979 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2272d-300e-4106-a9d7-58b7158aaf32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:51.914998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2272d-300e-4106-a9d7-58b7158aaf32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:51.916631 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2272d-300e-4106-a9d7-58b7158aaf32 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:51.926702 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9daa6c-1a36-4763-a589-da1eff62e344 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:51.931712 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 821/1633] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:51.933422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2272d-300e-4106-a9d7-58b7158aaf32 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:51.941333 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb446d-1047-4e22-8c92-0708989ca833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:51.942632 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb446d-1047-4e22-8c92-0708989ca833 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:51.943248 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb446d-1047-4e22-8c92-0708989ca833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:51.943799 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb446d-1047-4e22-8c92-0708989ca833 admin admin] PATH_INFO: `/v3/projects/d0423bd03d0b4c598ac4911bf46471e6/users/ecfc26485a344bd3b109d0f67d03700a/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:51.961167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2272d-300e-4106-a9d7-58b7158aaf32 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:51.975353 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a030f-7a69-4317-8055-8589a5b2053d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:51.977618 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a030f-7a69-4317-8055-8589a5b2053d admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:51.978294 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a030f-7a69-4317-8055-8589a5b2053d admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:51.978906 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a030f-7a69-4317-8055-8589a5b2053d admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:51.982358 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a030f-7a69-4317-8055-8589a5b2053d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:51.990875 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a030f-7a69-4317-8055-8589a5b2053d admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:51.995447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb446d-1047-4e22-8c92-0708989ca833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0423bd03d0b4c598ac4911bf46471e6, user_id=ecfc26485a344bd3b109d0f67d03700a, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:51.998727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb446d-1047-4e22-8c92-0708989ca833 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:52.007145 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:52.030583 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 822/1634] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:52.031684 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a030f-7a69-4317-8055-8589a5b2053d admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:52.069710 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a030f-7a69-4317-8055-8589a5b2053d admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:52.084347 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ff39c-67e8-4fc1-a65b-f03584225422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:52.086556 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ff39c-67e8-4fc1-a65b-f03584225422 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:52.086857 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ff39c-67e8-4fc1-a65b-f03584225422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:52.087169 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ff39c-67e8-4fc1-a65b-f03584225422 admin admin] PATH_INFO: `/v3/projects/6009eebfd4a84bc193cae411e2621ca2/users/f96aacb0268f4ea9b0d04d5cd0bb1d87/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:52.094547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 813/1635] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:23:51 2022] PUT /identity/v3/projects/d0423bd03d0b4c598ac4911bf46471e6/users/ecfc26485a344bd3b109d0f67d03700a/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:23:52.102635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb446d-1047-4e22-8c92-0708989ca833 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:52.130326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ff39c-67e8-4fc1-a65b-f03584225422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6009eebfd4a84bc193cae411e2621ca2, user_id=f96aacb0268f4ea9b0d04d5cd0bb1d87, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:52.133516 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ff39c-67e8-4fc1-a65b-f03584225422 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:52.196663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb446d-1047-4e22-8c92-0708989ca833 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:52.201586 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9dd46-24db-4455-9cfb-59584eaa5a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:52.202472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9dd46-24db-4455-9cfb-59584eaa5a9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:52.202754 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9dd46-24db-4455-9cfb-59584eaa5a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:52.203072 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9dd46-24db-4455-9cfb-59584eaa5a9c admin admin] PATH_INFO: `/v3/projects/cdc74866edd347b288deeeeb01e6f360/users/fb44b8ba8c07431a9badfc4a7cd92a74/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:52.219458 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9dd46-24db-4455-9cfb-59584eaa5a9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdc74866edd347b288deeeeb01e6f360, user_id=fb44b8ba8c07431a9badfc4a7cd92a74, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:52.222073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9dd46-24db-4455-9cfb-59584eaa5a9c admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:52.274712 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 823/1636] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:23:52 2022] PUT /identity/v3/projects/6009eebfd4a84bc193cae411e2621ca2/users/f96aacb0268f4ea9b0d04d5cd0bb1d87/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:23:52.274712 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ff39c-67e8-4fc1-a65b-f03584225422 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:52.340125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 814/1637] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:23:52 2022] PUT /identity/v3/projects/cdc74866edd347b288deeeeb01e6f360/users/fb44b8ba8c07431a9badfc4a7cd92a74/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:23:52.341847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f1fec-1a2b-4521-8ee5-45847e2c4ba6 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:52.342269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f1fec-1a2b-4521-8ee5-45847e2c4ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:52.342693 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f1fec-1a2b-4521-8ee5-45847e2c4ba6 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:52.345899 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 815/1638] 10.222.0.52 () {54 vars in 912 bytes} [Thu Jun 23 09:23:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:23:52.348615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f1fec-1a2b-4521-8ee5-45847e2c4ba6 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:52.366593 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ff39c-67e8-4fc1-a65b-f03584225422 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:52.480996 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f1fec-1a2b-4521-8ee5-45847e2c4ba6 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:52.485877 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec870e-1291-443d-b482-621ee6cc72f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:52.486787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec870e-1291-443d-b482-621ee6cc72f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:52.491345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec870e-1291-443d-b482-621ee6cc72f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:52.491345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec870e-1291-443d-b482-621ee6cc72f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:52.491345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec870e-1291-443d-b482-621ee6cc72f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:52.491700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932eef76-3f3a-4bbc-bdac-e6b6e856f1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:52.491700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932eef76-3f3a-4bbc-bdac-e6b6e856f1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:52.491700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932eef76-3f3a-4bbc-bdac-e6b6e856f1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:52.491700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932eef76-3f3a-4bbc-bdac-e6b6e856f1d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:52.492099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec870e-1291-443d-b482-621ee6cc72f1 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:52.492198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932eef76-3f3a-4bbc-bdac-e6b6e856f1d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:52.492280 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932eef76-3f3a-4bbc-bdac-e6b6e856f1d0 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:52.503595 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 824/1639] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:52.506074 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 816/1640] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:52.508048 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a1c58-9f79-4267-abe4-ca4a0fe98810 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:52.508048 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a1c58-9f79-4267-abe4-ca4a0fe98810 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:52.508302 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a1c58-9f79-4267-abe4-ca4a0fe98810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:52.509062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec870e-1291-443d-b482-621ee6cc72f1 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:52.524772 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec870e-1291-443d-b482-621ee6cc72f1 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:52.529347 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-7e7a1c58-9f79-4267-abe4-ca4a0fe98810 None None] MFA Rules not processed for user `ff025cb706d243778468ae4570374690`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:52.540370 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2ed26-6315-4e7e-a4e9-abc63f3d072f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:52.543095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2ed26-6315-4e7e-a4e9-abc63f3d072f admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:52.543095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2ed26-6315-4e7e-a4e9-abc63f3d072f admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:52.543095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2ed26-6315-4e7e-a4e9-abc63f3d072f admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:52.543095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee2ed26-6315-4e7e-a4e9-abc63f3d072f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:52.543482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-7e7a1c58-9f79-4267-abe4-ca4a0fe98810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:52.544503 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee2ed26-6315-4e7e-a4e9-abc63f3d072f admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:52.569404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 825/1641] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:23:52.571795 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2ed26-6315-4e7e-a4e9-abc63f3d072f admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:52.602717 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2ed26-6315-4e7e-a4e9-abc63f3d072f admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:52.613174 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79b6d9-14fd-453e-98d8-a1a40652e9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:52.614141 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79b6d9-14fd-453e-98d8-a1a40652e9dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:52.614484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79b6d9-14fd-453e-98d8-a1a40652e9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:52.614949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79b6d9-14fd-453e-98d8-a1a40652e9dc admin admin] PATH_INFO: `/v3/projects/d0423bd03d0b4c598ac4911bf46471e6/users/ecfc26485a344bd3b109d0f67d03700a/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:52.617311 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79b6d9-14fd-453e-98d8-a1a40652e9dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0423bd03d0b4c598ac4911bf46471e6, user_id=ecfc26485a344bd3b109d0f67d03700a, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:52.620244 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79b6d9-14fd-453e-98d8-a1a40652e9dc admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:52.640833 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 826/1642] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:23:52 2022] PUT /identity/v3/projects/d0423bd03d0b4c598ac4911bf46471e6/users/ecfc26485a344bd3b109d0f67d03700a/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:23:52.641688 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79b6d9-14fd-453e-98d8-a1a40652e9dc admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:52.695609 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 817/1643] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jun 23 09:23:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:52.697057 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a1c58-9f79-4267-abe4-ca4a0fe98810 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:52.707969 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a1c58-9f79-4267-abe4-ca4a0fe98810 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:52.721968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:52.782663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9b610-fa95-451c-8c76-42fea9c26d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:52.783636 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9b610-fa95-451c-8c76-42fea9c26d37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:52.783773 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79b6d9-14fd-453e-98d8-a1a40652e9dc admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:52.784108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9b610-fa95-451c-8c76-42fea9c26d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:52.784252 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9b610-fa95-451c-8c76-42fea9c26d37 admin admin] PATH_INFO: `/v3/projects/cdc74866edd347b288deeeeb01e6f360/users/fb44b8ba8c07431a9badfc4a7cd92a74/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:52.787206 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9b610-fa95-451c-8c76-42fea9c26d37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdc74866edd347b288deeeeb01e6f360, user_id=fb44b8ba8c07431a9badfc4a7cd92a74, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:52.789916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9b610-fa95-451c-8c76-42fea9c26d37 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:52.798625 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86bb91e-65ca-48d2-8168-a534093911e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:52.799588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86bb91e-65ca-48d2-8168-a534093911e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:52.803352 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86bb91e-65ca-48d2-8168-a534093911e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:52.803352 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86bb91e-65ca-48d2-8168-a534093911e3 admin admin] PATH_INFO: `/v3/projects/6009eebfd4a84bc193cae411e2621ca2/users/f96aacb0268f4ea9b0d04d5cd0bb1d87/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:52.806344 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86bb91e-65ca-48d2-8168-a534093911e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6009eebfd4a84bc193cae411e2621ca2, user_id=f96aacb0268f4ea9b0d04d5cd0bb1d87, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:52.809893 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86bb91e-65ca-48d2-8168-a534093911e3 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:52.827737 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 818/1644] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:23:52 2022] PUT /identity/v3/projects/cdc74866edd347b288deeeeb01e6f360/users/fb44b8ba8c07431a9badfc4a7cd92a74/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:23:52.829355 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188b51a-44ca-4cb9-9523-63c96a858ea4 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:52.834041 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188b51a-44ca-4cb9-9523-63c96a858ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:52.834041 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188b51a-44ca-4cb9-9523-63c96a858ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:52.840897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 827/1645] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:23:52 2022] PUT /identity/v3/projects/6009eebfd4a84bc193cae411e2621ca2/users/f96aacb0268f4ea9b0d04d5cd0bb1d87/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:23:52.843015 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86bb91e-65ca-48d2-8168-a534093911e3 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:52.872400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-c188b51a-44ca-4cb9-9523-63c96a858ea4 None None] MFA Rules not processed for user `ecfc26485a344bd3b109d0f67d03700a`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:52.874174 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-c188b51a-44ca-4cb9-9523-63c96a858ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:52.939685 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86bb91e-65ca-48d2-8168-a534093911e3 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:52.943935 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd2982c-7365-4bc9-b26c-6e72a66348fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:52.946257 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2982c-7365-4bc9-b26c-6e72a66348fb service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:52.946257 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2982c-7365-4bc9-b26c-6e72a66348fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:52.946257 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2982c-7365-4bc9-b26c-6e72a66348fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:52.948395 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:53.005886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 819/1646] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:52 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:53.017719 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94598866-c253-41f5-94e7-4a068be4a15e None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:53.017719 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94598866-c253-41f5-94e7-4a068be4a15e None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:53.017719 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94598866-c253-41f5-94e7-4a068be4a15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:53.029316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd2982c-7365-4bc9-b26c-6e72a66348fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:53.030843 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd2982c-7365-4bc9-b26c-6e72a66348fb service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:53.037244 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-94598866-c253-41f5-94e7-4a068be4a15e None None] MFA Rules not processed for user `fb44b8ba8c07431a9badfc4a7cd92a74`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:53.038441 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-94598866-c253-41f5-94e7-4a068be4a15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:53.043215 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 828/1647] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:23:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:23:53.044932 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e222f3-01d3-41c2-b846-9960a17599fa None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:53.045226 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e222f3-01d3-41c2-b846-9960a17599fa None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:53.045511 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e222f3-01d3-41c2-b846-9960a17599fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:53.086053 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-e7e222f3-01d3-41c2-b846-9960a17599fa None None] MFA Rules not processed for user `f96aacb0268f4ea9b0d04d5cd0bb1d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:53.087443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e7e222f3-01d3-41c2-b846-9960a17599fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:53.162004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 820/1648] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:53 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:53.162883 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94598866-c253-41f5-94e7-4a068be4a15e None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:53.184258 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94598866-c253-41f5-94e7-4a068be4a15e None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:53.197175 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb2624-e800-4e71-a8dd-b34ba18854d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:53.198837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb2624-e800-4e71-a8dd-b34ba18854d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:53.199349 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb2624-e800-4e71-a8dd-b34ba18854d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:53.200635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb2624-e800-4e71-a8dd-b34ba18854d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:53.246383 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 829/1649] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:53 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:53.252703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb2624-e800-4e71-a8dd-b34ba18854d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:53.257211 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb2624-e800-4e71-a8dd-b34ba18854d2 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:53.294876 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 821/1650] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:23:53 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:23:53.387421 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e222f3-01d3-41c2-b846-9960a17599fa None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:53.398854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e222f3-01d3-41c2-b846-9960a17599fa None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:53.546892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b24c5c-bae6-4630-aaba-7963f3cf0f5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:53.546892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b24c5c-bae6-4630-aaba-7963f3cf0f5a service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:53.546892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b24c5c-bae6-4630-aaba-7963f3cf0f5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:53.546892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b24c5c-bae6-4630-aaba-7963f3cf0f5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:53.550226 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b24c5c-bae6-4630-aaba-7963f3cf0f5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:53.550226 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b24c5c-bae6-4630-aaba-7963f3cf0f5a service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:53.550226 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 830/1651] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:23:53 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:23:53.654930 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb2624-e800-4e71-a8dd-b34ba18854d2 service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:53.667086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb2624-e800-4e71-a8dd-b34ba18854d2 service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:53.672572 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052fc43-0931-4c9f-9dd0-470cf004eacc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:53.673219 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052fc43-0931-4c9f-9dd0-470cf004eacc service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:53.673611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052fc43-0931-4c9f-9dd0-470cf004eacc service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:53.673943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052fc43-0931-4c9f-9dd0-470cf004eacc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:53.685200 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052fc43-0931-4c9f-9dd0-470cf004eacc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:53.686983 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052fc43-0931-4c9f-9dd0-470cf004eacc service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:53.700820 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 822/1652] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:23:53 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:23:53.928539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:54.323524 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b24c5c-bae6-4630-aaba-7963f3cf0f5a service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:54.323524 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b24c5c-bae6-4630-aaba-7963f3cf0f5a service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:54.323524 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18850004-6d3d-4754-8c51-4ebb43be2a93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:54.323524 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18850004-6d3d-4754-8c51-4ebb43be2a93 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:54.333342 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18850004-6d3d-4754-8c51-4ebb43be2a93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:54.333342 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18850004-6d3d-4754-8c51-4ebb43be2a93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:54.390304 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18850004-6d3d-4754-8c51-4ebb43be2a93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:54.392246 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18850004-6d3d-4754-8c51-4ebb43be2a93 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:54.404492 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 831/1653] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:23:54 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:23:54.538328 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:54.788420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c9471-f8d3-4b70-9078-69f60e2b3f3f None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:54.788701 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c9471-f8d3-4b70-9078-69f60e2b3f3f None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:54.788998 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c9471-f8d3-4b70-9078-69f60e2b3f3f None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:54.791271 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 823/1654] 10.222.0.52 () {54 vars in 961 bytes} [Thu Jun 23 09:23:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:23:54.802452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84d6ca-3084-43c9-9dfc-04ea634b6ba9 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:54.803028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84d6ca-3084-43c9-9dfc-04ea634b6ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:54.803447 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84d6ca-3084-43c9-9dfc-04ea634b6ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:54.828491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-cf84d6ca-3084-43c9-9dfc-04ea634b6ba9 None None] MFA Rules not processed for user `ff025cb706d243778468ae4570374690`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:54.830568 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-cf84d6ca-3084-43c9-9dfc-04ea634b6ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:54.857020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 832/1655] 10.222.0.52 () {58 vars in 1058 bytes} [Thu Jun 23 09:23:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:54.868121 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef7bbb-d0bf-43ac-829e-5e18293f3085 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:54.869462 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef7bbb-d0bf-43ac-829e-5e18293f3085 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:54.875180 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef7bbb-d0bf-43ac-829e-5e18293f3085 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:54.879673 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 824/1656] 10.222.0.52 () {54 vars in 912 bytes} [Thu Jun 23 09:23:54 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:23:54.883568 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b63a9b-268a-4867-9976-c3b2e44b35da None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:54.883876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b63a9b-268a-4867-9976-c3b2e44b35da None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:54.884165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b63a9b-268a-4867-9976-c3b2e44b35da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:54.909634 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-05b63a9b-268a-4867-9976-c3b2e44b35da None None] MFA Rules not processed for user `ff025cb706d243778468ae4570374690`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:54.911019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-05b63a9b-268a-4867-9976-c3b2e44b35da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:54.936314 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 833/1657] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jun 23 09:23:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:55.190557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b63a9b-268a-4867-9976-c3b2e44b35da None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:55.191077 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef7bbb-d0bf-43ac-829e-5e18293f3085 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:55.210204 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b63a9b-268a-4867-9976-c3b2e44b35da None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:55.219034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa3f24-d535-43e5-be7c-031610099162 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:55.220842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa3f24-d535-43e5-be7c-031610099162 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:55.222351 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa3f24-d535-43e5-be7c-031610099162 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:55.222841 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa3f24-d535-43e5-be7c-031610099162 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:55.339182 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa3f24-d535-43e5-be7c-031610099162 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:55.340866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa3f24-d535-43e5-be7c-031610099162 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:55.354662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 834/1658] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:23:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:23:55.436295 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef7bbb-d0bf-43ac-829e-5e18293f3085 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:55.442879 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa3f24-d535-43e5-be7c-031610099162 service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:55.445268 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b6249-9ad7-4a35-8a26-50dbef8d405e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:55.446276 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b6249-9ad7-4a35-8a26-50dbef8d405e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:55.446758 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b6249-9ad7-4a35-8a26-50dbef8d405e admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:55.447279 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b6249-9ad7-4a35-8a26-50dbef8d405e admin admin] PATH_INFO: `/v3/users/bc50bc409c7e401f9294202d641db3cf` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:55.450417 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b6249-9ad7-4a35-8a26-50dbef8d405e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc50bc409c7e401f9294202d641db3cf)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:55.452101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: WARNING py.warnings [None req-7e0b6249-9ad7-4a35-8a26-50dbef8d405e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 09:23:55.452101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: warnings.warn(msg) Jun 23 09:23:55.452101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: Jun 23 09:23:55.453347 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b6249-9ad7-4a35-8a26-50dbef8d405e admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:55.455845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa3f24-d535-43e5-be7c-031610099162 service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:55.476294 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a471044-e748-4755-a675-feecacc23453 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:55.478023 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a471044-e748-4755-a675-feecacc23453 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:55.481762 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a471044-e748-4755-a675-feecacc23453 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:55.483707 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a471044-e748-4755-a675-feecacc23453 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:55.532103 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a471044-e748-4755-a675-feecacc23453 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:55.538871 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a471044-e748-4755-a675-feecacc23453 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:55.570855 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 835/1659] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:23:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:23:55.816390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7e0b6249-9ad7-4a35-8a26-50dbef8d405e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc50bc409c7e401f9294202d641db3cf'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:23:55.846578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7e0b6249-9ad7-4a35-8a26-50dbef8d405e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc50bc409c7e401f9294202d641db3cf'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:23:55.849820 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7e0b6249-9ad7-4a35-8a26-50dbef8d405e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc50bc409c7e401f9294202d641db3cf'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:23:55.851186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7e0b6249-9ad7-4a35-8a26-50dbef8d405e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc50bc409c7e401f9294202d641db3cf'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:23:55.891264 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7e0b6249-9ad7-4a35-8a26-50dbef8d405e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc50bc409c7e401f9294202d641db3cf'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:23:55.969956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 825/1660] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:23:55 2022] DELETE /identity/v3/users/bc50bc409c7e401f9294202d641db3cf => generated 0 bytes in 781 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:23:55.978992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc08a3a-94e5-4dec-a41c-ffa9d9b6e8c3 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:55.979301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc08a3a-94e5-4dec-a41c-ffa9d9b6e8c3 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:55.979597 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc08a3a-94e5-4dec-a41c-ffa9d9b6e8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:56.028452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-3dc08a3a-94e5-4dec-a41c-ffa9d9b6e8c3 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:23:56.029774 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-3dc08a3a-94e5-4dec-a41c-ffa9d9b6e8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:56.203381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 836/1661] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:23:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:23:56.218868 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b6249-9ad7-4a35-8a26-50dbef8d405e admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:56.220621 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-7e0b6249-9ad7-4a35-8a26-50dbef8d405e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:56.298598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b6249-9ad7-4a35-8a26-50dbef8d405e admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:56.303480 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422f8bd-4959-4de7-9b12-52e3cf012491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:56.303772 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:56.304486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422f8bd-4959-4de7-9b12-52e3cf012491 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:56.304947 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422f8bd-4959-4de7-9b12-52e3cf012491 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:56.305380 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422f8bd-4959-4de7-9b12-52e3cf012491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:56.322415 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422f8bd-4959-4de7-9b12-52e3cf012491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:56.325940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422f8bd-4959-4de7-9b12-52e3cf012491 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:56.344327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 826/1662] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:23:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:23:57.098488 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc08a3a-94e5-4dec-a41c-ffa9d9b6e8c3 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:57.098488 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-3dc08a3a-94e5-4dec-a41c-ffa9d9b6e8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:57.124994 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc08a3a-94e5-4dec-a41c-ffa9d9b6e8c3 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:57.160100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468876dd-be80-4e6a-b8fa-0b4f3b597c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:57.165791 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468876dd-be80-4e6a-b8fa-0b4f3b597c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:57.165791 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468876dd-be80-4e6a-b8fa-0b4f3b597c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:57.165791 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468876dd-be80-4e6a-b8fa-0b4f3b597c31 admin admin] PATH_INFO: `/v3/projects/75ca8c6ca93741a085c55058a543b245` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:57.165791 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468876dd-be80-4e6a-b8fa-0b4f3b597c31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75ca8c6ca93741a085c55058a543b245)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:57.165791 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: WARNING py.warnings [None req-468876dd-be80-4e6a-b8fa-0b4f3b597c31 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 09:23:57.165791 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: warnings.warn(msg) Jun 23 09:23:57.165791 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: Jun 23 09:23:57.166708 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468876dd-be80-4e6a-b8fa-0b4f3b597c31 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:57.433969 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-468876dd-be80-4e6a-b8fa-0b4f3b597c31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75ca8c6ca93741a085c55058a543b245', 'request_id': 'req-468876dd-be80-4e6a-b8fa-0b4f3b597c31'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:23:57.666787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-468876dd-be80-4e6a-b8fa-0b4f3b597c31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75ca8c6ca93741a085c55058a543b245', 'request_id': 'req-468876dd-be80-4e6a-b8fa-0b4f3b597c31'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:23:57.674855 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-468876dd-be80-4e6a-b8fa-0b4f3b597c31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75ca8c6ca93741a085c55058a543b245', 'request_id': 'req-468876dd-be80-4e6a-b8fa-0b4f3b597c31'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:23:57.786972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-468876dd-be80-4e6a-b8fa-0b4f3b597c31 admin admin] The token cache is being invalidate because project 75ca8c6ca93741a085c55058a543b245 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:23:57.787561 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-468876dd-be80-4e6a-b8fa-0b4f3b597c31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:23:57.788389 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-468876dd-be80-4e6a-b8fa-0b4f3b597c31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:23:57.790137 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 837/1663] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:23:57 2022] DELETE /identity/v3/projects/75ca8c6ca93741a085c55058a543b245 => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:23:57.794406 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422f8bd-4959-4de7-9b12-52e3cf012491 service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:57.801077 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:23:57.806059 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422f8bd-4959-4de7-9b12-52e3cf012491 service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:57.874264 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c3eca-bbb2-4e30-b195-4bcc159d6182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:57.875296 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c3eca-bbb2-4e30-b195-4bcc159d6182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:57.875761 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c3eca-bbb2-4e30-b195-4bcc159d6182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:57.876193 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c3eca-bbb2-4e30-b195-4bcc159d6182 admin admin] PATH_INFO: `/v3/users/ecfc26485a344bd3b109d0f67d03700a` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:57.878061 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c3eca-bbb2-4e30-b195-4bcc159d6182 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecfc26485a344bd3b109d0f67d03700a)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:57.880770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c3eca-bbb2-4e30-b195-4bcc159d6182 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:58.207582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-816c3eca-bbb2-4e30-b195-4bcc159d6182 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecfc26485a344bd3b109d0f67d03700a'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:23:58.216832 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-816c3eca-bbb2-4e30-b195-4bcc159d6182 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecfc26485a344bd3b109d0f67d03700a'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:23:58.219306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-816c3eca-bbb2-4e30-b195-4bcc159d6182 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecfc26485a344bd3b109d0f67d03700a'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:23:58.220752 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-816c3eca-bbb2-4e30-b195-4bcc159d6182 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecfc26485a344bd3b109d0f67d03700a'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:23:58.240164 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-816c3eca-bbb2-4e30-b195-4bcc159d6182 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecfc26485a344bd3b109d0f67d03700a'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:23:58.357119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 827/1664] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:23:57 2022] DELETE /identity/v3/users/ecfc26485a344bd3b109d0f67d03700a => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:23:59.546611 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468876dd-be80-4e6a-b8fa-0b4f3b597c31 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:59.546611 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-468876dd-be80-4e6a-b8fa-0b4f3b597c31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:59.567322 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468876dd-be80-4e6a-b8fa-0b4f3b597c31 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:23:59.692634 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c278380-7a3f-468f-9dd0-47e751941592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:23:59.693537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c278380-7a3f-468f-9dd0-47e751941592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:23:59.693819 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c278380-7a3f-468f-9dd0-47e751941592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:23:59.700601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c278380-7a3f-468f-9dd0-47e751941592 admin admin] PATH_INFO: `/v3/projects/d0423bd03d0b4c598ac4911bf46471e6` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:23:59.702286 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c278380-7a3f-468f-9dd0-47e751941592 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0423bd03d0b4c598ac4911bf46471e6)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:23:59.706719 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c278380-7a3f-468f-9dd0-47e751941592 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:23:59.924332 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-5c278380-7a3f-468f-9dd0-47e751941592 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0423bd03d0b4c598ac4911bf46471e6', 'request_id': 'req-5c278380-7a3f-468f-9dd0-47e751941592'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:23:59.949095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-5c278380-7a3f-468f-9dd0-47e751941592 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0423bd03d0b4c598ac4911bf46471e6', 'request_id': 'req-5c278380-7a3f-468f-9dd0-47e751941592'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:23:59.951130 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c3eca-bbb2-4e30-b195-4bcc159d6182 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:23:59.951130 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-816c3eca-bbb2-4e30-b195-4bcc159d6182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:23:59.960452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-5c278380-7a3f-468f-9dd0-47e751941592 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0423bd03d0b4c598ac4911bf46471e6', 'request_id': 'req-5c278380-7a3f-468f-9dd0-47e751941592'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:23:59.970761 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c3eca-bbb2-4e30-b195-4bcc159d6182 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:00.069919 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-5c278380-7a3f-468f-9dd0-47e751941592 admin admin] The token cache is being invalidate because project d0423bd03d0b4c598ac4911bf46471e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:24:00.070223 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-5c278380-7a3f-468f-9dd0-47e751941592 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:00.072108 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-5c278380-7a3f-468f-9dd0-47e751941592 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:00.074775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 838/1665] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:23:59 2022] DELETE /identity/v3/projects/d0423bd03d0b4c598ac4911bf46471e6 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:00.079936 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c278380-7a3f-468f-9dd0-47e751941592 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:00.081619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-5c278380-7a3f-468f-9dd0-47e751941592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:00.090626 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:00.144410 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384bf77-4eb6-45a0-a40e-2775978cefee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:00.146347 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384bf77-4eb6-45a0-a40e-2775978cefee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:00.148534 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384bf77-4eb6-45a0-a40e-2775978cefee admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:00.149017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384bf77-4eb6-45a0-a40e-2775978cefee admin admin] PATH_INFO: `/v3/users/4570d1fead8c4328a99cd58a02867c45` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:00.152759 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f384bf77-4eb6-45a0-a40e-2775978cefee admin admin] RBAC: Authorizing `identity:delete_user(user_id=4570d1fead8c4328a99cd58a02867c45)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:00.159945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f384bf77-4eb6-45a0-a40e-2775978cefee admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:00.201834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c278380-7a3f-468f-9dd0-47e751941592 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:00.215095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2eb346-bbe0-4bca-a562-0d905492561e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:00.220685 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2eb346-bbe0-4bca-a562-0d905492561e service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:00.223118 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2eb346-bbe0-4bca-a562-0d905492561e service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:00.226845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2eb346-bbe0-4bca-a562-0d905492561e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:00.230521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-aa2eb346-bbe0-4bca-a562-0d905492561e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:00.390245 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2eb346-bbe0-4bca-a562-0d905492561e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:00.392896 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2eb346-bbe0-4bca-a562-0d905492561e service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:00.428520 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f384bf77-4eb6-45a0-a40e-2775978cefee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4570d1fead8c4328a99cd58a02867c45'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:00.440990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f384bf77-4eb6-45a0-a40e-2775978cefee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4570d1fead8c4328a99cd58a02867c45'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:00.443563 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f384bf77-4eb6-45a0-a40e-2775978cefee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4570d1fead8c4328a99cd58a02867c45'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:00.444244 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f384bf77-4eb6-45a0-a40e-2775978cefee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4570d1fead8c4328a99cd58a02867c45'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:00.446960 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 839/1666] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:00 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:00.453688 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add80b42-3b36-440f-8faf-6e3c2db5cb6c None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:00.460948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add80b42-3b36-440f-8faf-6e3c2db5cb6c None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:00.460948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add80b42-3b36-440f-8faf-6e3c2db5cb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:00.481256 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f384bf77-4eb6-45a0-a40e-2775978cefee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4570d1fead8c4328a99cd58a02867c45'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:00.550952 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-add80b42-3b36-440f-8faf-6e3c2db5cb6c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:00.552807 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-add80b42-3b36-440f-8faf-6e3c2db5cb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:00.582440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 840/1667] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:00.585383 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb72d0c-0fde-423d-872d-e966759153ab None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:00.585927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb72d0c-0fde-423d-872d-e966759153ab None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:00.586247 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:00.587023 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb72d0c-0fde-423d-872d-e966759153ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:00.595121 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 828/1668] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:23:59 2022] DELETE /identity/v3/users/4570d1fead8c4328a99cd58a02867c45 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:00.599337 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb30a36-40e1-4c5d-b88a-f113c09931d6 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:00.599856 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb30a36-40e1-4c5d-b88a-f113c09931d6 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:00.600319 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb30a36-40e1-4c5d-b88a-f113c09931d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:00.618323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-2bb72d0c-0fde-423d-872d-e966759153ab None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:00.619725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-2bb72d0c-0fde-423d-872d-e966759153ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:00.655942 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-ffb30a36-40e1-4c5d-b88a-f113c09931d6 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:00.659346 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ffb30a36-40e1-4c5d-b88a-f113c09931d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:00.771739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 841/1669] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:00.772812 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb72d0c-0fde-423d-872d-e966759153ab None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:00.788143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb72d0c-0fde-423d-872d-e966759153ab None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:00.822029 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b59cf7-a90e-4f8a-8cd3-75b95c0a2f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:00.822738 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b59cf7-a90e-4f8a-8cd3-75b95c0a2f48 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:00.823063 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b59cf7-a90e-4f8a-8cd3-75b95c0a2f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:00.823356 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b59cf7-a90e-4f8a-8cd3-75b95c0a2f48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:00.827485 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b59cf7-a90e-4f8a-8cd3-75b95c0a2f48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:24:00.831193 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b59cf7-a90e-4f8a-8cd3-75b95c0a2f48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:00.836703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b59cf7-a90e-4f8a-8cd3-75b95c0a2f48 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:00.860494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 842/1670] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:24:00.863893 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b59cf7-a90e-4f8a-8cd3-75b95c0a2f48 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:00.864959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 829/1671] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:24:00.876079 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb30a36-40e1-4c5d-b88a-f113c09931d6 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:00.877613 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ffb30a36-40e1-4c5d-b88a-f113c09931d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:00.883501 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b59cf7-a90e-4f8a-8cd3-75b95c0a2f48 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:00.888424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db953752-a669-48fd-9050-a9d8dddf2168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:00.889113 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db953752-a669-48fd-9050-a9d8dddf2168 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:00.889468 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db953752-a669-48fd-9050-a9d8dddf2168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:00.889764 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db953752-a669-48fd-9050-a9d8dddf2168 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:00.890476 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db953752-a669-48fd-9050-a9d8dddf2168 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:00.893155 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db953752-a669-48fd-9050-a9d8dddf2168 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:01.009666 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 843/1672] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:24:00 2022] POST /identity/v3/projects => generated 369 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:01.021229 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4850ea-8544-4cf5-97d2-3161ec901a50 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:01.027722 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4850ea-8544-4cf5-97d2-3161ec901a50 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:01.028154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4850ea-8544-4cf5-97d2-3161ec901a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:01.086210 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb30a36-40e1-4c5d-b88a-f113c09931d6 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:01.153357 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-2b4850ea-8544-4cf5-97d2-3161ec901a50 None None] MFA Rules not processed for user `30bc3dba44ed4d14a67e56aa640bf9ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:01.163128 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-2b4850ea-8544-4cf5-97d2-3161ec901a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:01.300739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073823f-1879-4946-90cb-868c6452d640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:01.301752 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073823f-1879-4946-90cb-868c6452d640 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:01.302257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073823f-1879-4946-90cb-868c6452d640 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:01.302744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073823f-1879-4946-90cb-868c6452d640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:01.393916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 844/1673] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:01.395207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4850ea-8544-4cf5-97d2-3161ec901a50 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:01.465021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073823f-1879-4946-90cb-868c6452d640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:01.470236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073823f-1879-4946-90cb-868c6452d640 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:01.492394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 830/1674] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:01.493340 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073823f-1879-4946-90cb-868c6452d640 service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:01.494825 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5073823f-1879-4946-90cb-868c6452d640 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:01.600431 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4850ea-8544-4cf5-97d2-3161ec901a50 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:01.622555 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469b454-dba8-46a8-98f4-d5149776d460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:01.623703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469b454-dba8-46a8-98f4-d5149776d460 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:01.623998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469b454-dba8-46a8-98f4-d5149776d460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:01.624282 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469b454-dba8-46a8-98f4-d5149776d460 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:01.629692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469b454-dba8-46a8-98f4-d5149776d460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:01.638879 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469b454-dba8-46a8-98f4-d5149776d460 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:01.695956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073823f-1879-4946-90cb-868c6452d640 service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:01.709542 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60381bc-4814-49ba-b83a-48c37effcf41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:01.712887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60381bc-4814-49ba-b83a-48c37effcf41 service nova] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:01.712887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60381bc-4814-49ba-b83a-48c37effcf41 service nova] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:01.712887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60381bc-4814-49ba-b83a-48c37effcf41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:01.731180 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60381bc-4814-49ba-b83a-48c37effcf41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:01.733151 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60381bc-4814-49ba-b83a-48c37effcf41 service nova] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:01.745150 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 831/1675] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:01.786898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 845/1676] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:24:01 2022] POST /identity/v3/users => generated 285 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:01.791740 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60381bc-4814-49ba-b83a-48c37effcf41 service nova] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:01.811563 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60381bc-4814-49ba-b83a-48c37effcf41 service nova] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:01.817853 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb2a20-043e-42bc-b26d-4cf34283f434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:01.818467 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb2a20-043e-42bc-b26d-4cf34283f434 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:01.819426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb2a20-043e-42bc-b26d-4cf34283f434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:01.819721 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb2a20-043e-42bc-b26d-4cf34283f434 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:01.820476 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb2a20-043e-42bc-b26d-4cf34283f434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:01.823255 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb2a20-043e-42bc-b26d-4cf34283f434 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:01.845649 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469b454-dba8-46a8-98f4-d5149776d460 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:01.855010 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 832/1677] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:01.857717 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469b454-dba8-46a8-98f4-d5149776d460 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:01.862677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74d52e-3dea-4af4-8b8c-d4c13256c92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:01.863140 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74d52e-3dea-4af4-8b8c-d4c13256c92b admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:01.863433 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74d52e-3dea-4af4-8b8c-d4c13256c92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:01.863714 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74d52e-3dea-4af4-8b8c-d4c13256c92b admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:01.864657 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74d52e-3dea-4af4-8b8c-d4c13256c92b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:01.866935 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74d52e-3dea-4af4-8b8c-d4c13256c92b admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:01.883467 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb2a20-043e-42bc-b26d-4cf34283f434 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:01.883901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 846/1678] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:24:01.894780 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb2a20-043e-42bc-b26d-4cf34283f434 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:01.918725 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fc311-218d-4b9a-bf88-0456a2cf1711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:01.922002 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fc311-218d-4b9a-bf88-0456a2cf1711 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:01.922213 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fc311-218d-4b9a-bf88-0456a2cf1711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:01.922508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fc311-218d-4b9a-bf88-0456a2cf1711 admin admin] PATH_INFO: `/v3/projects/fb544cadeb2c4b07ab43ec22cc7ec638/users/b556c588b8da47de9b8c58515a91d8e2/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:01.993605 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51fc311-218d-4b9a-bf88-0456a2cf1711 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb544cadeb2c4b07ab43ec22cc7ec638, user_id=b556c588b8da47de9b8c58515a91d8e2, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:01.997324 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51fc311-218d-4b9a-bf88-0456a2cf1711 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:02.116989 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 833/1679] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:01 2022] PUT /identity/v3/projects/fb544cadeb2c4b07ab43ec22cc7ec638/users/b556c588b8da47de9b8c58515a91d8e2/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:02.137108 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80369a21-a61c-4a67-8a11-d8232ce0068c None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:02.137420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80369a21-a61c-4a67-8a11-d8232ce0068c None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:02.138224 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80369a21-a61c-4a67-8a11-d8232ce0068c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:02.191652 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-80369a21-a61c-4a67-8a11-d8232ce0068c None None] MFA Rules not processed for user `b556c588b8da47de9b8c58515a91d8e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:02.193092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-80369a21-a61c-4a67-8a11-d8232ce0068c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:02.325381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 847/1680] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:02.329884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fc311-218d-4b9a-bf88-0456a2cf1711 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:02.448499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fc311-218d-4b9a-bf88-0456a2cf1711 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:02.461192 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d1265-e2eb-4b62-a8ac-a071dc19dad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:02.470250 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d1265-e2eb-4b62-a8ac-a071dc19dad9 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:02.476099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d1265-e2eb-4b62-a8ac-a071dc19dad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:02.476099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d1265-e2eb-4b62-a8ac-a071dc19dad9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:02.476099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d1265-e2eb-4b62-a8ac-a071dc19dad9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:02.478258 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d1265-e2eb-4b62-a8ac-a071dc19dad9 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:02.598370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 834/1681] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:24:02 2022] POST /identity/v3/projects => generated 367 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:02.603602 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80369a21-a61c-4a67-8a11-d8232ce0068c None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:02.623069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80369a21-a61c-4a67-8a11-d8232ce0068c None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:02.755226 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715add2a-4b20-483a-bf21-54346ada0543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:02.756866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715add2a-4b20-483a-bf21-54346ada0543 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:02.757079 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715add2a-4b20-483a-bf21-54346ada0543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:02.757349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715add2a-4b20-483a-bf21-54346ada0543 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:02.758248 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715add2a-4b20-483a-bf21-54346ada0543 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:02.760619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715add2a-4b20-483a-bf21-54346ada0543 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:02.801293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:02.889176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 848/1682] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:24:02 2022] POST /identity/v3/users => generated 290 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:02.898700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d1265-e2eb-4b62-a8ac-a071dc19dad9 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:02.913063 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d1265-e2eb-4b62-a8ac-a071dc19dad9 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:02.933777 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3915e75-c100-417c-bf99-7552dc8b2c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:02.934267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3915e75-c100-417c-bf99-7552dc8b2c5e admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:02.935102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3915e75-c100-417c-bf99-7552dc8b2c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:02.935392 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3915e75-c100-417c-bf99-7552dc8b2c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:02.936470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3915e75-c100-417c-bf99-7552dc8b2c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:02.945520 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715add2a-4b20-483a-bf21-54346ada0543 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:02.947743 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-715add2a-4b20-483a-bf21-54346ada0543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:02.948932 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3915e75-c100-417c-bf99-7552dc8b2c5e admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:02.971963 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715add2a-4b20-483a-bf21-54346ada0543 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:02.988113 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd3804-a2a8-434a-8588-d41efe5df3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:02.989628 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd3804-a2a8-434a-8588-d41efe5df3d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:02.990181 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd3804-a2a8-434a-8588-d41efe5df3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:02.991202 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd3804-a2a8-434a-8588-d41efe5df3d0 admin admin] PATH_INFO: `/v3/projects/ad2e6e427876444390f75d4cc8080347` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:02.995162 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd3804-a2a8-434a-8588-d41efe5df3d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad2e6e427876444390f75d4cc8080347)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:02.997881 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd3804-a2a8-434a-8588-d41efe5df3d0 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:02.999343 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 835/1683] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:03.000506 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3915e75-c100-417c-bf99-7552dc8b2c5e admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:03.033422 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3915e75-c100-417c-bf99-7552dc8b2c5e admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:03.055647 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3d860-08c4-4794-8b3e-20f5e4f69be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:03.056828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3d860-08c4-4794-8b3e-20f5e4f69be4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:03.057266 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3d860-08c4-4794-8b3e-20f5e4f69be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:03.057691 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3d860-08c4-4794-8b3e-20f5e4f69be4 admin admin] PATH_INFO: `/v3/projects/7008c27e6d2f4a91aee1caa05156f04b/users/afb2f596561740e5a0b8f1d05ea01552/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:03.099948 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea3d860-08c4-4794-8b3e-20f5e4f69be4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7008c27e6d2f4a91aee1caa05156f04b, user_id=afb2f596561740e5a0b8f1d05ea01552, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:03.103142 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea3d860-08c4-4794-8b3e-20f5e4f69be4 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:03.142668 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c2fd3804-a2a8-434a-8588-d41efe5df3d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad2e6e427876444390f75d4cc8080347', 'request_id': 'req-c2fd3804-a2a8-434a-8588-d41efe5df3d0'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:03.173423 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c2fd3804-a2a8-434a-8588-d41efe5df3d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad2e6e427876444390f75d4cc8080347', 'request_id': 'req-c2fd3804-a2a8-434a-8588-d41efe5df3d0'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:03.216271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c2fd3804-a2a8-434a-8588-d41efe5df3d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad2e6e427876444390f75d4cc8080347', 'request_id': 'req-c2fd3804-a2a8-434a-8588-d41efe5df3d0'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:03.235811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 836/1684] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:02 2022] PUT /identity/v3/projects/7008c27e6d2f4a91aee1caa05156f04b/users/afb2f596561740e5a0b8f1d05ea01552/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:03.239910 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3d860-08c4-4794-8b3e-20f5e4f69be4 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:03.328067 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c2fd3804-a2a8-434a-8588-d41efe5df3d0 admin admin] The token cache is being invalidate because project ad2e6e427876444390f75d4cc8080347 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:24:03.328602 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c2fd3804-a2a8-434a-8588-d41efe5df3d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:03.330100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c2fd3804-a2a8-434a-8588-d41efe5df3d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:03.336824 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 849/1685] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:24:02 2022] DELETE /identity/v3/projects/ad2e6e427876444390f75d4cc8080347 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:03.338671 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d0c67-1fda-40f3-ad53-9e0d18b15288 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:03.340116 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d0c67-1fda-40f3-ad53-9e0d18b15288 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:03.340567 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d0c67-1fda-40f3-ad53-9e0d18b15288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:03.366978 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-df2d0c67-1fda-40f3-ad53-9e0d18b15288 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:03.369286 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-df2d0c67-1fda-40f3-ad53-9e0d18b15288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:03.376661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3d860-08c4-4794-8b3e-20f5e4f69be4 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:03.389401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5e078-c55a-4de0-af3d-efaf35931a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:03.391846 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5e078-c55a-4de0-af3d-efaf35931a89 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:03.393129 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5e078-c55a-4de0-af3d-efaf35931a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:03.395211 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5e078-c55a-4de0-af3d-efaf35931a89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:03.396775 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5e078-c55a-4de0-af3d-efaf35931a89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:03.400536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 850/1686] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:03.402651 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08157aa5-7b2e-4319-b45a-4acfce1296b8 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:03.403109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08157aa5-7b2e-4319-b45a-4acfce1296b8 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:03.403586 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08157aa5-7b2e-4319-b45a-4acfce1296b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:03.404258 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5e078-c55a-4de0-af3d-efaf35931a89 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:03.452880 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:03.456909 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 837/1687] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:03.462259 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5e078-c55a-4de0-af3d-efaf35931a89 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:03.464409 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-39b5e078-c55a-4de0-af3d-efaf35931a89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:03.469292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-08157aa5-7b2e-4319-b45a-4acfce1296b8 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:03.472615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-08157aa5-7b2e-4319-b45a-4acfce1296b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:03.489679 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5e078-c55a-4de0-af3d-efaf35931a89 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:03.499034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 851/1688] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:03.503426 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08157aa5-7b2e-4319-b45a-4acfce1296b8 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:03.521023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf24e3-5e73-4844-8b42-c86a4ac58279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:03.521023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf24e3-5e73-4844-8b42-c86a4ac58279 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:03.521023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf24e3-5e73-4844-8b42-c86a4ac58279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:03.521023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf24e3-5e73-4844-8b42-c86a4ac58279 admin admin] PATH_INFO: `/v3/projects/7008c27e6d2f4a91aee1caa05156f04b/users/afb2f596561740e5a0b8f1d05ea01552/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:03.525614 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08157aa5-7b2e-4319-b45a-4acfce1296b8 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:03.527632 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf24e3-5e73-4844-8b42-c86a4ac58279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7008c27e6d2f4a91aee1caa05156f04b, user_id=afb2f596561740e5a0b8f1d05ea01552, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:03.533184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf24e3-5e73-4844-8b42-c86a4ac58279 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:03.551839 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d424f-2e2f-4c51-9f7a-226c242460c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:03.552693 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d424f-2e2f-4c51-9f7a-226c242460c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:03.552963 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d424f-2e2f-4c51-9f7a-226c242460c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:03.553446 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d424f-2e2f-4c51-9f7a-226c242460c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:03.554811 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d424f-2e2f-4c51-9f7a-226c242460c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:24:03.554811 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d424f-2e2f-4c51-9f7a-226c242460c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:03.556695 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d424f-2e2f-4c51-9f7a-226c242460c8 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:03.559153 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 838/1689] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:03 2022] PUT /identity/v3/projects/7008c27e6d2f4a91aee1caa05156f04b/users/afb2f596561740e5a0b8f1d05ea01552/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:03.573038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013d211-b8d3-4c2f-83ec-ac8c4888c53b None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:03.573448 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013d211-b8d3-4c2f-83ec-ac8c4888c53b None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:03.577857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013d211-b8d3-4c2f-83ec-ac8c4888c53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:03.583930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 852/1690] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:24:03.596218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d424f-2e2f-4c51-9f7a-226c242460c8 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:03.612457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d424f-2e2f-4c51-9f7a-226c242460c8 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:03.655851 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-c013d211-b8d3-4c2f-83ec-ac8c4888c53b None None] MFA Rules not processed for user `afb2f596561740e5a0b8f1d05ea01552`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:03.657027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-c013d211-b8d3-4c2f-83ec-ac8c4888c53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:03.799733 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e01053-0ae7-4653-9b76-96a6a585a22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:03.800359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e01053-0ae7-4653-9b76-96a6a585a22d admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:03.800649 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e01053-0ae7-4653-9b76-96a6a585a22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:03.800927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e01053-0ae7-4653-9b76-96a6a585a22d admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:03.801644 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e01053-0ae7-4653-9b76-96a6a585a22d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:03.803632 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e01053-0ae7-4653-9b76-96a6a585a22d admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:03.818007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 839/1691] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:03 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:03.819958 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734ddc7-7f13-4716-bda9-06e58f6d1e0d None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:03.820482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734ddc7-7f13-4716-bda9-06e58f6d1e0d None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:03.821481 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734ddc7-7f13-4716-bda9-06e58f6d1e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:03.910122 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-f734ddc7-7f13-4716-bda9-06e58f6d1e0d None None] MFA Rules not processed for user `29fe5fb181e2452388d24fd0b7aa5e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:03.913774 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-f734ddc7-7f13-4716-bda9-06e58f6d1e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:04.029380 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 853/1692] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:24:03 2022] POST /identity/v3/projects => generated 363 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:04.031966 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e01053-0ae7-4653-9b76-96a6a585a22d admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:04.036043 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b5e01053-0ae7-4653-9b76-96a6a585a22d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:04.076289 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 840/1693] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:04.076289 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734ddc7-7f13-4716-bda9-06e58f6d1e0d None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:04.100356 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734ddc7-7f13-4716-bda9-06e58f6d1e0d None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:04.251786 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e01053-0ae7-4653-9b76-96a6a585a22d admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:04.271059 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bcf97-65b9-4b72-926e-e792af2f4c09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:04.271713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bcf97-65b9-4b72-926e-e792af2f4c09 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:04.273121 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bcf97-65b9-4b72-926e-e792af2f4c09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:04.273468 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bcf97-65b9-4b72-926e-e792af2f4c09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:04.369533 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37e702-895c-41e5-b28d-54e83cf87b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:04.370549 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37e702-895c-41e5-b28d-54e83cf87b36 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:04.370885 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37e702-895c-41e5-b28d-54e83cf87b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:04.371435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37e702-895c-41e5-b28d-54e83cf87b36 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:04.372721 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37e702-895c-41e5-b28d-54e83cf87b36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:04.375219 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37e702-895c-41e5-b28d-54e83cf87b36 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:04.492672 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 841/1694] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:24:04 2022] POST /identity/v3/users => generated 282 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:04.494216 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906edb3-4e2d-4a84-ab68-351eec04e587 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:04.494512 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906edb3-4e2d-4a84-ab68-351eec04e587 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:04.494790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906edb3-4e2d-4a84-ab68-351eec04e587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:04.505100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:04.554348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bcf97-65b9-4b72-926e-e792af2f4c09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:04.554891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-d906edb3-4e2d-4a84-ab68-351eec04e587 None None] MFA Rules not processed for user `29fe5fb181e2452388d24fd0b7aa5e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:04.555913 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bcf97-65b9-4b72-926e-e792af2f4c09 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:04.558815 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d906edb3-4e2d-4a84-ab68-351eec04e587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:04.599364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 854/1695] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:04 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:04.600563 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bcf97-65b9-4b72-926e-e792af2f4c09 service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:04.628521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bcf97-65b9-4b72-926e-e792af2f4c09 service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:04.643965 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a53cd-205c-412e-9fa4-3c95e9e7440a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:04.647445 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a53cd-205c-412e-9fa4-3c95e9e7440a admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:04.648096 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a53cd-205c-412e-9fa4-3c95e9e7440a admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:04.648453 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a53cd-205c-412e-9fa4-3c95e9e7440a admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:04.650873 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a53cd-205c-412e-9fa4-3c95e9e7440a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:04.659845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a53cd-205c-412e-9fa4-3c95e9e7440a admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:04.694551 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 855/1696] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:04.699710 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a53cd-205c-412e-9fa4-3c95e9e7440a admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:04.718168 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 842/1697] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:04.719754 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a53cd-205c-412e-9fa4-3c95e9e7440a admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:04.730724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906edb3-4e2d-4a84-ab68-351eec04e587 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:04.733347 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d906edb3-4e2d-4a84-ab68-351eec04e587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:04.734330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc14916-04fc-41aa-bc7c-9e65ca4f99c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:04.735329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc14916-04fc-41aa-bc7c-9e65ca4f99c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:04.735788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc14916-04fc-41aa-bc7c-9e65ca4f99c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:04.736295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc14916-04fc-41aa-bc7c-9e65ca4f99c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:04.737210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc14916-04fc-41aa-bc7c-9e65ca4f99c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:04.740571 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc14916-04fc-41aa-bc7c-9e65ca4f99c0 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:04.775490 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 856/1698] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:04.781088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc14916-04fc-41aa-bc7c-9e65ca4f99c0 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:04.802397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc14916-04fc-41aa-bc7c-9e65ca4f99c0 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:04.820976 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf16ce-0989-46a0-961c-a4ceeeea40a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:04.822377 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf16ce-0989-46a0-961c-a4ceeeea40a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:04.822671 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf16ce-0989-46a0-961c-a4ceeeea40a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:04.823235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf16ce-0989-46a0-961c-a4ceeeea40a5 admin admin] PATH_INFO: `/v3/projects/ced9fa9aa54d42dda557bb17d14fe971/users/bbd74757c4fe4e6ebb1f3845d9ef11eb/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:04.879892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf16ce-0989-46a0-961c-a4ceeeea40a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ced9fa9aa54d42dda557bb17d14fe971, user_id=bbd74757c4fe4e6ebb1f3845d9ef11eb, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:04.886209 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf16ce-0989-46a0-961c-a4ceeeea40a5 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:04.925739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906edb3-4e2d-4a84-ab68-351eec04e587 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:04.944090 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571731d-0f42-4a8a-82b5-65a01a5dd5c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:04.945101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571731d-0f42-4a8a-82b5-65a01a5dd5c0 service nova] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:04.945580 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571731d-0f42-4a8a-82b5-65a01a5dd5c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:04.946169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571731d-0f42-4a8a-82b5-65a01a5dd5c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:04.968897 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571731d-0f42-4a8a-82b5-65a01a5dd5c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:04.971457 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571731d-0f42-4a8a-82b5-65a01a5dd5c0 service nova] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:04.985160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 843/1699] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:24:04.987381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571731d-0f42-4a8a-82b5-65a01a5dd5c0 service nova] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:05.001055 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571731d-0f42-4a8a-82b5-65a01a5dd5c0 service nova] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:05.005236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91baf75-e1a1-4f57-8b10-5578efcaff4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:05.006116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91baf75-e1a1-4f57-8b10-5578efcaff4c service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:05.006565 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91baf75-e1a1-4f57-8b10-5578efcaff4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:05.007077 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91baf75-e1a1-4f57-8b10-5578efcaff4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:05.011186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-c91baf75-e1a1-4f57-8b10-5578efcaff4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:05.047034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 857/1700] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:04 2022] PUT /identity/v3/projects/ced9fa9aa54d42dda557bb17d14fe971/users/bbd74757c4fe4e6ebb1f3845d9ef11eb/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:05.048741 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff80dc-0a8f-4a25-8ae3-c7d94eac5e40 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:05.049045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff80dc-0a8f-4a25-8ae3-c7d94eac5e40 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:05.049333 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff80dc-0a8f-4a25-8ae3-c7d94eac5e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:05.103902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-89ff80dc-0a8f-4a25-8ae3-c7d94eac5e40 None None] MFA Rules not processed for user `6e7f2d909d45471fb0b97ddb724cb195`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:05.105264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-89ff80dc-0a8f-4a25-8ae3-c7d94eac5e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:05.127486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91baf75-e1a1-4f57-8b10-5578efcaff4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:05.131103 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91baf75-e1a1-4f57-8b10-5578efcaff4c service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:05.159175 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 844/1701] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:04 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:05.160760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e88be2-5fb3-43fc-a7d8-d40c5e277602 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:05.161065 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e88be2-5fb3-43fc-a7d8-d40c5e277602 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:05.161349 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e88be2-5fb3-43fc-a7d8-d40c5e277602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:05.209343 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-f5e88be2-5fb3-43fc-a7d8-d40c5e277602 None None] MFA Rules not processed for user `bbd74757c4fe4e6ebb1f3845d9ef11eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:05.210618 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-f5e88be2-5fb3-43fc-a7d8-d40c5e277602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:05.283824 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 858/1702] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:05.296202 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636b09b-c49b-4a6e-8fe1-36a5f30a9ce1 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:05.298617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636b09b-c49b-4a6e-8fe1-36a5f30a9ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:05.298617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636b09b-c49b-4a6e-8fe1-36a5f30a9ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:05.339692 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 845/1703] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:05 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:05.347053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e88be2-5fb3-43fc-a7d8-d40c5e277602 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:05.348729 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-f5e88be2-5fb3-43fc-a7d8-d40c5e277602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:05.366894 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-3636b09b-c49b-4a6e-8fe1-36a5f30a9ce1 None None] MFA Rules not processed for user `6e7f2d909d45471fb0b97ddb724cb195`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:05.377878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-3636b09b-c49b-4a6e-8fe1-36a5f30a9ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:05.406970 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 859/1704] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:05.421420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636b09b-c49b-4a6e-8fe1-36a5f30a9ce1 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:05.425493 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-3636b09b-c49b-4a6e-8fe1-36a5f30a9ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:05.527124 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:05.608404 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e88be2-5fb3-43fc-a7d8-d40c5e277602 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:05.614607 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd52fad-7262-41bc-a5df-fefdb4106721 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:05.616409 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd52fad-7262-41bc-a5df-fefdb4106721 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:05.616409 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd52fad-7262-41bc-a5df-fefdb4106721 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:05.616409 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd52fad-7262-41bc-a5df-fefdb4106721 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:05.635523 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd52fad-7262-41bc-a5df-fefdb4106721 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:05.637167 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd52fad-7262-41bc-a5df-fefdb4106721 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:05.668410 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636b09b-c49b-4a6e-8fe1-36a5f30a9ce1 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:05.671196 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 846/1705] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:05 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:24:05.676194 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f602a-bbaf-4bec-9c28-df442a2788f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:05.676194 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f602a-bbaf-4bec-9c28-df442a2788f7 service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:05.676444 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f602a-bbaf-4bec-9c28-df442a2788f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:05.676731 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f602a-bbaf-4bec-9c28-df442a2788f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:05.686080 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f602a-bbaf-4bec-9c28-df442a2788f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:05.687494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f602a-bbaf-4bec-9c28-df442a2788f7 service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:05.698477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 860/1706] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:05.784419 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:05.875460 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a05098-3330-4d46-936f-99e153c53fac None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:05.875752 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a05098-3330-4d46-936f-99e153c53fac None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:05.876020 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a05098-3330-4d46-936f-99e153c53fac None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:05.877713 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 847/1707] 10.222.0.52 () {54 vars in 912 bytes} [Thu Jun 23 09:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:24:05.884569 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027a5ea-d1ce-42b4-86df-cd949cb594d7 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:05.884890 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027a5ea-d1ce-42b4-86df-cd949cb594d7 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:05.885351 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027a5ea-d1ce-42b4-86df-cd949cb594d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:05.939987 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-a027a5ea-d1ce-42b4-86df-cd949cb594d7 None None] MFA Rules not processed for user `daed809f511a43ed8d3f88eba10466b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:05.941228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-a027a5ea-d1ce-42b4-86df-cd949cb594d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:06.014055 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 861/1708] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jun 23 09:24:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:06.027028 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a05098-3330-4d46-936f-99e153c53fac None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:06.029479 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a8a05098-3330-4d46-936f-99e153c53fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:06.044425 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a05098-3330-4d46-936f-99e153c53fac None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:06.049315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e36f6-559f-4f23-90b7-d12f3ca27b1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:06.050048 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e36f6-559f-4f23-90b7-d12f3ca27b1b service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:06.050341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e36f6-559f-4f23-90b7-d12f3ca27b1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:06.050620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e36f6-559f-4f23-90b7-d12f3ca27b1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:06.060626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e36f6-559f-4f23-90b7-d12f3ca27b1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:06.062112 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e36f6-559f-4f23-90b7-d12f3ca27b1b service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:06.075142 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 848/1709] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:06.351730 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431986a-d3e7-412c-8bfa-5201d6aa730b None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:06.352646 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431986a-d3e7-412c-8bfa-5201d6aa730b None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:06.352646 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431986a-d3e7-412c-8bfa-5201d6aa730b None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:06.354318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 862/1710] 10.222.0.52 () {54 vars in 912 bytes} [Thu Jun 23 09:24:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:24:06.366156 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0e125-19e4-419f-a4c7-c20608d975d6 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:06.371438 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0e125-19e4-419f-a4c7-c20608d975d6 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:06.371438 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0e125-19e4-419f-a4c7-c20608d975d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:06.413521 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-b1b0e125-19e4-419f-a4c7-c20608d975d6 None None] MFA Rules not processed for user `daed809f511a43ed8d3f88eba10466b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:06.414809 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-b1b0e125-19e4-419f-a4c7-c20608d975d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:06.458993 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 849/1711] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jun 23 09:24:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:06.491885 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431986a-d3e7-412c-8bfa-5201d6aa730b None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:06.497346 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-3431986a-d3e7-412c-8bfa-5201d6aa730b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:06.526569 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431986a-d3e7-412c-8bfa-5201d6aa730b None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:06.539759 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a24e9b-d8c7-4a46-a2ef-f06983fc343c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:06.539759 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a24e9b-d8c7-4a46-a2ef-f06983fc343c service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:06.539759 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a24e9b-d8c7-4a46-a2ef-f06983fc343c service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:06.539759 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a24e9b-d8c7-4a46-a2ef-f06983fc343c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:06.560733 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a24e9b-d8c7-4a46-a2ef-f06983fc343c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:06.562338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a24e9b-d8c7-4a46-a2ef-f06983fc343c service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:06.585195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 863/1712] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:06.769112 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87107f-bfbd-4f04-bf34-dbbb96ecd438 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:06.769659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87107f-bfbd-4f04-bf34-dbbb96ecd438 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:06.769659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87107f-bfbd-4f04-bf34-dbbb96ecd438 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:06.771842 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 850/1713] 10.222.0.52 () {54 vars in 960 bytes} [Thu Jun 23 09:24:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:24:06.777303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e5a2d-d8a7-4bc4-b5bb-fa960cbd01c9 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:06.778861 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e5a2d-d8a7-4bc4-b5bb-fa960cbd01c9 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:06.779671 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e5a2d-d8a7-4bc4-b5bb-fa960cbd01c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:06.809597 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-e38e5a2d-d8a7-4bc4-b5bb-fa960cbd01c9 None None] MFA Rules not processed for user `98a63b20ceca40e6bceb56eda8ae9686`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:06.811116 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e38e5a2d-d8a7-4bc4-b5bb-fa960cbd01c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:06.846035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686275f7-5bf3-4ca4-bbb8-4943e9613133 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:06.846306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686275f7-5bf3-4ca4-bbb8-4943e9613133 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:06.846592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686275f7-5bf3-4ca4-bbb8-4943e9613133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:06.871208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-686275f7-5bf3-4ca4-bbb8-4943e9613133 None None] MFA Rules not processed for user `29fe5fb181e2452388d24fd0b7aa5e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:06.873277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-686275f7-5bf3-4ca4-bbb8-4943e9613133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:06.916019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 864/1714] 10.222.0.52 () {58 vars in 1057 bytes} [Thu Jun 23 09:24:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:24:06.926824 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e5a2d-d8a7-4bc4-b5bb-fa960cbd01c9 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:06.949451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e5a2d-d8a7-4bc4-b5bb-fa960cbd01c9 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:06.962165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c0458-b8a1-43d6-ad4f-1f1f2e3c5d23 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63b20ceca40e6bceb56eda8ae9686', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:06.963338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c0458-b8a1-43d6-ad4f-1f1f2e3c5d23 service placement] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:06.963841 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c0458-b8a1-43d6-ad4f-1f1f2e3c5d23 service placement] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:06.966201 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c0458-b8a1-43d6-ad4f-1f1f2e3c5d23 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:06.973177 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-555c0458-b8a1-43d6-ad4f-1f1f2e3c5d23 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:06.980854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 851/1715] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:06.996625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:07.016297 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c0458-b8a1-43d6-ad4f-1f1f2e3c5d23 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:07.025800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c0458-b8a1-43d6-ad4f-1f1f2e3c5d23 service placement] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:07.032284 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 865/1716] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:07.238532 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacc80e-aa83-4f62-a106-a33c9ca9f41e None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:07.238903 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacc80e-aa83-4f62-a106-a33c9ca9f41e None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:07.239242 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacc80e-aa83-4f62-a106-a33c9ca9f41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:07.266173 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-deacc80e-aa83-4f62-a106-a33c9ca9f41e None None] MFA Rules not processed for user `30bc3dba44ed4d14a67e56aa640bf9ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:07.268145 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-deacc80e-aa83-4f62-a106-a33c9ca9f41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:07.384981 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 852/1717] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:08.161362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:08.658412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:10.741808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:11.366661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:13.362024 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ffbad7-982d-4b31-a5ec-272d6bea4ae4 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:13.362503 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ffbad7-982d-4b31-a5ec-272d6bea4ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:13.362581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ffbad7-982d-4b31-a5ec-272d6bea4ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:13.393607 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-b7ffbad7-982d-4b31-a5ec-272d6bea4ae4 None None] MFA Rules not processed for user `cfbaba84402d4c31bafcd12133d74c03`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:13.394890 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b7ffbad7-982d-4b31-a5ec-272d6bea4ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:13.602020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 866/1718] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jun 23 09:24:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:14.115037 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacc80e-aa83-4f62-a106-a33c9ca9f41e None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:14.147726 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ffbad7-982d-4b31-a5ec-272d6bea4ae4 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:14.162373 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b7ffbad7-982d-4b31-a5ec-272d6bea4ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:14.212357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacc80e-aa83-4f62-a106-a33c9ca9f41e None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:14.237346 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915a5f8-b91a-46dd-9b52-42197d64ec45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:14.238553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915a5f8-b91a-46dd-9b52-42197d64ec45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:14.239238 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915a5f8-b91a-46dd-9b52-42197d64ec45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:14.239669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915a5f8-b91a-46dd-9b52-42197d64ec45 admin admin] PATH_INFO: `/v3/users/b556c588b8da47de9b8c58515a91d8e2` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:14.241674 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915a5f8-b91a-46dd-9b52-42197d64ec45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b556c588b8da47de9b8c58515a91d8e2)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:14.245102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915a5f8-b91a-46dd-9b52-42197d64ec45 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:14.310777 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ffbad7-982d-4b31-a5ec-272d6bea4ae4 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:14.319027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773591e-acbf-4239-a719-baf7015b6d46 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b1331b4f0924bb8b137a22a33c9d130', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:14.320209 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773591e-acbf-4239-a719-baf7015b6d46 service swift] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:14.320964 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773591e-acbf-4239-a719-baf7015b6d46 service swift] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:14.321403 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773591e-acbf-4239-a719-baf7015b6d46 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:14.339205 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5773591e-acbf-4239-a719-baf7015b6d46 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:14.339205 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5773591e-acbf-4239-a719-baf7015b6d46 service swift] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:14.351954 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 867/1719] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jun 23 09:24:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 09:24:14.474250 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:14.508948 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1915a5f8-b91a-46dd-9b52-42197d64ec45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b556c588b8da47de9b8c58515a91d8e2'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:14.539933 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1915a5f8-b91a-46dd-9b52-42197d64ec45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b556c588b8da47de9b8c58515a91d8e2'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:14.542279 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1915a5f8-b91a-46dd-9b52-42197d64ec45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b556c588b8da47de9b8c58515a91d8e2'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:14.545275 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1915a5f8-b91a-46dd-9b52-42197d64ec45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b556c588b8da47de9b8c58515a91d8e2'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:14.598623 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1915a5f8-b91a-46dd-9b52-42197d64ec45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b556c588b8da47de9b8c58515a91d8e2'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:14.725402 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 853/1720] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:24:14 2022] DELETE /identity/v3/users/b556c588b8da47de9b8c58515a91d8e2 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:14.730104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23e3c0-86a9-4211-b848-41ac8642c27d None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:14.730604 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23e3c0-86a9-4211-b848-41ac8642c27d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:14.730714 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23e3c0-86a9-4211-b848-41ac8642c27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:14.752148 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-fb23e3c0-86a9-4211-b848-41ac8642c27d None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:14.753435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-fb23e3c0-86a9-4211-b848-41ac8642c27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:14.847928 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:14.876002 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 868/1721] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:14.920013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915a5f8-b91a-46dd-9b52-42197d64ec45 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:14.921530 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-1915a5f8-b91a-46dd-9b52-42197d64ec45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:15.088279 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915a5f8-b91a-46dd-9b52-42197d64ec45 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:15.103695 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083e607-863f-4394-82ee-c4c33cecccd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:15.104379 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083e607-863f-4394-82ee-c4c33cecccd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:15.104673 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083e607-863f-4394-82ee-c4c33cecccd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:15.104958 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083e607-863f-4394-82ee-c4c33cecccd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:15.133440 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2083e607-863f-4394-82ee-c4c33cecccd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:15.137381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2083e607-863f-4394-82ee-c4c33cecccd5 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:15.172767 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 854/1722] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:15.879325 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23e3c0-86a9-4211-b848-41ac8642c27d None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:15.880884 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-fb23e3c0-86a9-4211-b848-41ac8642c27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:15.894984 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23e3c0-86a9-4211-b848-41ac8642c27d None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:15.901083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab4acf-040c-4874-8bd5-c04e71eb9c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:15.902308 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab4acf-040c-4874-8bd5-c04e71eb9c34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:15.902560 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab4acf-040c-4874-8bd5-c04e71eb9c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:15.902840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab4acf-040c-4874-8bd5-c04e71eb9c34 admin admin] PATH_INFO: `/v3/projects/fb544cadeb2c4b07ab43ec22cc7ec638` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:15.909220 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cab4acf-040c-4874-8bd5-c04e71eb9c34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb544cadeb2c4b07ab43ec22cc7ec638)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:15.911243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cab4acf-040c-4874-8bd5-c04e71eb9c34 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:16.041597 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-9cab4acf-040c-4874-8bd5-c04e71eb9c34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb544cadeb2c4b07ab43ec22cc7ec638', 'request_id': 'req-9cab4acf-040c-4874-8bd5-c04e71eb9c34'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:16.049381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-9cab4acf-040c-4874-8bd5-c04e71eb9c34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb544cadeb2c4b07ab43ec22cc7ec638', 'request_id': 'req-9cab4acf-040c-4874-8bd5-c04e71eb9c34'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:16.055245 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-9cab4acf-040c-4874-8bd5-c04e71eb9c34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb544cadeb2c4b07ab43ec22cc7ec638', 'request_id': 'req-9cab4acf-040c-4874-8bd5-c04e71eb9c34'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:16.160907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-9cab4acf-040c-4874-8bd5-c04e71eb9c34 admin admin] The token cache is being invalidate because project fb544cadeb2c4b07ab43ec22cc7ec638 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:24:16.165515 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-9cab4acf-040c-4874-8bd5-c04e71eb9c34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:16.165515 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-9cab4acf-040c-4874-8bd5-c04e71eb9c34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:16.165515 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 869/1723] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:24:15 2022] DELETE /identity/v3/projects/fb544cadeb2c4b07ab43ec22cc7ec638 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:16.167488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083e607-863f-4394-82ee-c4c33cecccd5 service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:16.177447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083e607-863f-4394-82ee-c4c33cecccd5 service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:16.233607 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72837224-beab-457f-96b6-7ffb0e918f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:16.234491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72837224-beab-457f-96b6-7ffb0e918f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:16.234786 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72837224-beab-457f-96b6-7ffb0e918f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:16.235149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72837224-beab-457f-96b6-7ffb0e918f24 admin admin] PATH_INFO: `/v3/users/afb2f596561740e5a0b8f1d05ea01552` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:16.236352 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72837224-beab-457f-96b6-7ffb0e918f24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afb2f596561740e5a0b8f1d05ea01552)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:16.238287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72837224-beab-457f-96b6-7ffb0e918f24 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:16.465420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-72837224-beab-457f-96b6-7ffb0e918f24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afb2f596561740e5a0b8f1d05ea01552'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:16.470271 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:16.473162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-72837224-beab-457f-96b6-7ffb0e918f24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afb2f596561740e5a0b8f1d05ea01552'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:16.473830 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-72837224-beab-457f-96b6-7ffb0e918f24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afb2f596561740e5a0b8f1d05ea01552'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:16.474446 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-72837224-beab-457f-96b6-7ffb0e918f24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afb2f596561740e5a0b8f1d05ea01552'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:16.486603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-72837224-beab-457f-96b6-7ffb0e918f24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afb2f596561740e5a0b8f1d05ea01552'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:16.635039 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 855/1724] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:24:16 2022] DELETE /identity/v3/users/afb2f596561740e5a0b8f1d05ea01552 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:17.348970 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab4acf-040c-4874-8bd5-c04e71eb9c34 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:17.351084 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-9cab4acf-040c-4874-8bd5-c04e71eb9c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:17.373898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab4acf-040c-4874-8bd5-c04e71eb9c34 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:17.453974 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d68a3-a06a-4e2e-a178-3e856f85c5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:17.455480 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d68a3-a06a-4e2e-a178-3e856f85c5b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:17.455993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d68a3-a06a-4e2e-a178-3e856f85c5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:17.456538 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d68a3-a06a-4e2e-a178-3e856f85c5b3 admin admin] PATH_INFO: `/v3/projects/7008c27e6d2f4a91aee1caa05156f04b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:17.458392 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d68a3-a06a-4e2e-a178-3e856f85c5b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7008c27e6d2f4a91aee1caa05156f04b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:17.460536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d68a3-a06a-4e2e-a178-3e856f85c5b3 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:17.596817 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ff7d68a3-a06a-4e2e-a178-3e856f85c5b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7008c27e6d2f4a91aee1caa05156f04b', 'request_id': 'req-ff7d68a3-a06a-4e2e-a178-3e856f85c5b3'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:17.604247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ff7d68a3-a06a-4e2e-a178-3e856f85c5b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7008c27e6d2f4a91aee1caa05156f04b', 'request_id': 'req-ff7d68a3-a06a-4e2e-a178-3e856f85c5b3'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:17.620662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ff7d68a3-a06a-4e2e-a178-3e856f85c5b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7008c27e6d2f4a91aee1caa05156f04b', 'request_id': 'req-ff7d68a3-a06a-4e2e-a178-3e856f85c5b3'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:17.767370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce908c05-0241-4e81-a329-9b6d9652aed7 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:17.767896 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce908c05-0241-4e81-a329-9b6d9652aed7 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:17.767994 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce908c05-0241-4e81-a329-9b6d9652aed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:17.817012 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-ce908c05-0241-4e81-a329-9b6d9652aed7 None None] MFA Rules not processed for user `daed809f511a43ed8d3f88eba10466b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:17.817012 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ce908c05-0241-4e81-a329-9b6d9652aed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:17.880687 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 856/1725] 10.222.0.55 () {58 vars in 1015 bytes} [Thu Jun 23 09:24:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:24:17.915405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce908c05-0241-4e81-a329-9b6d9652aed7 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:17.916825 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ce908c05-0241-4e81-a329-9b6d9652aed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:17.928317 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce908c05-0241-4e81-a329-9b6d9652aed7 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:17.932850 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94685e-9585-46be-afa8-2adf0977d6e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:17.933634 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94685e-9585-46be-afa8-2adf0977d6e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:17.933985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94685e-9585-46be-afa8-2adf0977d6e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:17.934864 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94685e-9585-46be-afa8-2adf0977d6e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:17.943850 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f94685e-9585-46be-afa8-2adf0977d6e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:17.945415 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f94685e-9585-46be-afa8-2adf0977d6e3 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:17.954877 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 857/1726] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:24:17.965752 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94685e-9585-46be-afa8-2adf0977d6e3 service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:17.987512 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94685e-9585-46be-afa8-2adf0977d6e3 service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:17.996107 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019de915-69f9-42b9-aaa5-e613b752e79b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:17.997296 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019de915-69f9-42b9-aaa5-e613b752e79b service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:17.997602 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019de915-69f9-42b9-aaa5-e613b752e79b service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:17.997884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019de915-69f9-42b9-aaa5-e613b752e79b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:18.006972 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019de915-69f9-42b9-aaa5-e613b752e79b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:18.008914 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019de915-69f9-42b9-aaa5-e613b752e79b service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:18.029006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 858/1727] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:18.031015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019de915-69f9-42b9-aaa5-e613b752e79b service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:18.034789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-019de915-69f9-42b9-aaa5-e613b752e79b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:18.063127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019de915-69f9-42b9-aaa5-e613b752e79b service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:18.077539 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36a8c5-73eb-42ba-ae57-e844810665ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:18.078302 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36a8c5-73eb-42ba-ae57-e844810665ed service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:18.078659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36a8c5-73eb-42ba-ae57-e844810665ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:18.079043 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36a8c5-73eb-42ba-ae57-e844810665ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:18.092234 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36a8c5-73eb-42ba-ae57-e844810665ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:18.093878 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36a8c5-73eb-42ba-ae57-e844810665ed service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:18.107413 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 859/1728] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:18.108717 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36a8c5-73eb-42ba-ae57-e844810665ed service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:18.132173 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36a8c5-73eb-42ba-ae57-e844810665ed service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:18.140546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2480f-e4b0-4cce-ba9c-eff31692a721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:18.141383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2480f-e4b0-4cce-ba9c-eff31692a721 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:18.141820 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2480f-e4b0-4cce-ba9c-eff31692a721 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:18.142306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2480f-e4b0-4cce-ba9c-eff31692a721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:18.164160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c2480f-e4b0-4cce-ba9c-eff31692a721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:18.166020 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c2480f-e4b0-4cce-ba9c-eff31692a721 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:18.175340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ff7d68a3-a06a-4e2e-a178-3e856f85c5b3 admin admin] The token cache is being invalidate because project 7008c27e6d2f4a91aee1caa05156f04b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:24:18.175999 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ff7d68a3-a06a-4e2e-a178-3e856f85c5b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:18.181408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ff7d68a3-a06a-4e2e-a178-3e856f85c5b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:18.187083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 870/1729] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:24:17 2022] DELETE /identity/v3/projects/7008c27e6d2f4a91aee1caa05156f04b => generated 0 bytes in 838 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:18.189154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 860/1730] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:18.191839 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5a6f3-4cc4-44e0-a849-995943e02554 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:18.192135 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5a6f3-4cc4-44e0-a849-995943e02554 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:18.195292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5a6f3-4cc4-44e0-a849-995943e02554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:18.262704 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-dec5a6f3-4cc4-44e0-a849-995943e02554 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:18.265451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-dec5a6f3-4cc4-44e0-a849-995943e02554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:18.278637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:18.395244 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 871/1731] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:18.396497 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4ff82-91a0-4b8f-96ed-c61114127efd None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:18.396850 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4ff82-91a0-4b8f-96ed-c61114127efd None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:18.397222 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4ff82-91a0-4b8f-96ed-c61114127efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:18.431998 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-0af4ff82-91a0-4b8f-96ed-c61114127efd None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:18.436299 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-0af4ff82-91a0-4b8f-96ed-c61114127efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:18.456487 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5a6f3-4cc4-44e0-a849-995943e02554 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:18.469594 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 861/1732] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:24:18.475371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5a6f3-4cc4-44e0-a849-995943e02554 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:18.504598 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9949719-ac1a-41ba-a8f3-f2ff95ea70d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:18.510421 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9949719-ac1a-41ba-a8f3-f2ff95ea70d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:18.511036 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9949719-ac1a-41ba-a8f3-f2ff95ea70d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:18.517345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9949719-ac1a-41ba-a8f3-f2ff95ea70d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:18.520943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9949719-ac1a-41ba-a8f3-f2ff95ea70d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:24:18.520943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9949719-ac1a-41ba-a8f3-f2ff95ea70d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:18.520943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9949719-ac1a-41ba-a8f3-f2ff95ea70d5 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:18.586965 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 872/1733] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:24:18.590189 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4ff82-91a0-4b8f-96ed-c61114127efd None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:18.607622 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4ff82-91a0-4b8f-96ed-c61114127efd None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:18.617930 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e425ffe-c8d9-49a5-8411-675e42b5e1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:18.618786 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e425ffe-c8d9-49a5-8411-675e42b5e1b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:18.619931 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e425ffe-c8d9-49a5-8411-675e42b5e1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:18.620208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e425ffe-c8d9-49a5-8411-675e42b5e1b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:18.621041 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e425ffe-c8d9-49a5-8411-675e42b5e1b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:18.623889 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e425ffe-c8d9-49a5-8411-675e42b5e1b0 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:18.745588 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 862/1734] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:24:18 2022] POST /identity/v3/projects => generated 391 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:18.746735 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9949719-ac1a-41ba-a8f3-f2ff95ea70d5 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:18.770228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9949719-ac1a-41ba-a8f3-f2ff95ea70d5 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:18.973114 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da467b-8285-4b33-bb3d-d973de38ed7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:18.983253 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da467b-8285-4b33-bb3d-d973de38ed7b admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:18.983253 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da467b-8285-4b33-bb3d-d973de38ed7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:18.983253 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da467b-8285-4b33-bb3d-d973de38ed7b admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:18.983253 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da467b-8285-4b33-bb3d-d973de38ed7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:18.983253 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da467b-8285-4b33-bb3d-d973de38ed7b admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:19.114369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 873/1735] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:24:18 2022] POST /identity/v3/users => generated 296 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:19.118187 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e425ffe-c8d9-49a5-8411-675e42b5e1b0 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:19.147086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e425ffe-c8d9-49a5-8411-675e42b5e1b0 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:19.185195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9915ee-c277-4e45-9c34-9974c3ef76d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:19.187855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9915ee-c277-4e45-9c34-9974c3ef76d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:19.188342 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9915ee-c277-4e45-9c34-9974c3ef76d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:19.189090 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9915ee-c277-4e45-9c34-9974c3ef76d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:19.196973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9915ee-c277-4e45-9c34-9974c3ef76d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:19.200598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9915ee-c277-4e45-9c34-9974c3ef76d7 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:19.227383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 863/1736] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:19.230316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da467b-8285-4b33-bb3d-d973de38ed7b admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:19.268323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da467b-8285-4b33-bb3d-d973de38ed7b admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:19.285346 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d0103-c283-4018-afb4-d4886944bb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:19.286515 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d0103-c283-4018-afb4-d4886944bb7d admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:19.286820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d0103-c283-4018-afb4-d4886944bb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:19.287124 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d0103-c283-4018-afb4-d4886944bb7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:19.288025 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d0103-c283-4018-afb4-d4886944bb7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:19.292157 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d0103-c283-4018-afb4-d4886944bb7d admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:19.321885 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 874/1737] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:24:19.326139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9915ee-c277-4e45-9c34-9974c3ef76d7 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:19.344603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9915ee-c277-4e45-9c34-9974c3ef76d7 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:19.357700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93c5c5-4109-4d64-b171-489a60802b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:19.359747 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93c5c5-4109-4d64-b171-489a60802b07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:19.360450 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93c5c5-4109-4d64-b171-489a60802b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:19.360984 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93c5c5-4109-4d64-b171-489a60802b07 admin admin] PATH_INFO: `/v3/projects/cee70b6e066040cc852b69193661f0d6/users/368d1e8cf88943b3a05729daeff4cb8e/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:19.389996 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93c5c5-4109-4d64-b171-489a60802b07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee70b6e066040cc852b69193661f0d6, user_id=368d1e8cf88943b3a05729daeff4cb8e, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:19.394367 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93c5c5-4109-4d64-b171-489a60802b07 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:19.533716 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 864/1738] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:19 2022] PUT /identity/v3/projects/cee70b6e066040cc852b69193661f0d6/users/368d1e8cf88943b3a05729daeff4cb8e/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:19.545082 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e5861-9730-4859-adcc-0950c62de36b None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:19.545546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e5861-9730-4859-adcc-0950c62de36b None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:19.546059 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e5861-9730-4859-adcc-0950c62de36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:19.617920 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-7f8e5861-9730-4859-adcc-0950c62de36b None None] MFA Rules not processed for user `368d1e8cf88943b3a05729daeff4cb8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:19.622878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7f8e5861-9730-4859-adcc-0950c62de36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:19.788197 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 875/1739] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:19.790163 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93c5c5-4109-4d64-b171-489a60802b07 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:19.872380 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:20.026663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93c5c5-4109-4d64-b171-489a60802b07 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:20.043452 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5e9d9-074f-4a54-a876-411edd0e68c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:20.044679 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5e9d9-074f-4a54-a876-411edd0e68c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:20.044968 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5e9d9-074f-4a54-a876-411edd0e68c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:20.045247 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5e9d9-074f-4a54-a876-411edd0e68c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:20.046225 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5e9d9-074f-4a54-a876-411edd0e68c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:20.048095 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5e9d9-074f-4a54-a876-411edd0e68c9 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:20.230791 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 865/1740] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:24:19 2022] POST /identity/v3/projects => generated 393 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:20.234058 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e5861-9730-4859-adcc-0950c62de36b None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:20.247533 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e5861-9730-4859-adcc-0950c62de36b None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:20.413991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68691a3-3287-42ea-ac00-36eb5376a156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:20.415722 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68691a3-3287-42ea-ac00-36eb5376a156 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:20.416232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68691a3-3287-42ea-ac00-36eb5376a156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:20.416676 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68691a3-3287-42ea-ac00-36eb5376a156 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:20.417545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68691a3-3287-42ea-ac00-36eb5376a156 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:20.420075 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68691a3-3287-42ea-ac00-36eb5376a156 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:20.560650 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 876/1741] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:24:20 2022] POST /identity/v3/users => generated 303 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:20.564427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5e9d9-074f-4a54-a876-411edd0e68c9 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:20.581578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5e9d9-074f-4a54-a876-411edd0e68c9 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:20.589295 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1df252b-a350-4b6e-8f17-5d3b7805498c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:20.591673 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df252b-a350-4b6e-8f17-5d3b7805498c admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:20.592147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df252b-a350-4b6e-8f17-5d3b7805498c admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:20.592589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df252b-a350-4b6e-8f17-5d3b7805498c admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:20.593683 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1df252b-a350-4b6e-8f17-5d3b7805498c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:20.595670 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1df252b-a350-4b6e-8f17-5d3b7805498c admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:20.616264 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 866/1742] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:20.619876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68691a3-3287-42ea-ac00-36eb5376a156 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:20.653713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68691a3-3287-42ea-ac00-36eb5376a156 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:20.679088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e770b960-6168-4e8d-83ca-0207b881b73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:20.680003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770b960-6168-4e8d-83ca-0207b881b73a admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:20.680474 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770b960-6168-4e8d-83ca-0207b881b73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:20.680923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770b960-6168-4e8d-83ca-0207b881b73a admin admin] PATH_INFO: `/v3/projects/e18c722103c144378f8498bbd8c11f82/users/3c93baa0ed244642998a63289e775d6d/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:20.742636 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e770b960-6168-4e8d-83ca-0207b881b73a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e18c722103c144378f8498bbd8c11f82, user_id=3c93baa0ed244642998a63289e775d6d, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:20.750958 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e770b960-6168-4e8d-83ca-0207b881b73a admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:20.921270 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 877/1743] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:20 2022] PUT /identity/v3/projects/e18c722103c144378f8498bbd8c11f82/users/3c93baa0ed244642998a63289e775d6d/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:20.925008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1df252b-a350-4b6e-8f17-5d3b7805498c admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:20.938880 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1df252b-a350-4b6e-8f17-5d3b7805498c admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:20.968884 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db6774-4cfd-4eeb-9263-eebc86393bac None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:20.969325 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db6774-4cfd-4eeb-9263-eebc86393bac None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:20.969719 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db6774-4cfd-4eeb-9263-eebc86393bac None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:20.971720 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 878/1744] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jun 23 09:24:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:24:21.113751 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32109e-c727-4cd9-bab1-02dcf7894465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:21.119081 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32109e-c727-4cd9-bab1-02dcf7894465 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:21.119445 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32109e-c727-4cd9-bab1-02dcf7894465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:21.119740 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32109e-c727-4cd9-bab1-02dcf7894465 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:21.120547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32109e-c727-4cd9-bab1-02dcf7894465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:21.126797 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32109e-c727-4cd9-bab1-02dcf7894465 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:21.129889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e57645-802f-400b-8a76-533a0f1b3ba1 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:21.130188 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e57645-802f-400b-8a76-533a0f1b3ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:21.130470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e57645-802f-400b-8a76-533a0f1b3ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:21.193108 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-c0e57645-802f-400b-8a76-533a0f1b3ba1 None None] MFA Rules not processed for user `ff025cb706d243778468ae4570374690`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:21.206732 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-c0e57645-802f-400b-8a76-533a0f1b3ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:21.211553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 867/1745] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:21.213122 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32109e-c727-4cd9-bab1-02dcf7894465 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:21.225683 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32109e-c727-4cd9-bab1-02dcf7894465 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:21.231933 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeef62-33b3-40f0-bd17-ecde24af101a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:21.234582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeef62-33b3-40f0-bd17-ecde24af101a admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:21.234582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeef62-33b3-40f0-bd17-ecde24af101a admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:21.235283 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeef62-33b3-40f0-bd17-ecde24af101a admin admin] PATH_INFO: `/v3/projects/e18c722103c144378f8498bbd8c11f82/users/3c93baa0ed244642998a63289e775d6d/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:21.239144 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaeef62-33b3-40f0-bd17-ecde24af101a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e18c722103c144378f8498bbd8c11f82, user_id=3c93baa0ed244642998a63289e775d6d, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:21.245697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaeef62-33b3-40f0-bd17-ecde24af101a admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:21.301608 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 868/1746] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:21 2022] PUT /identity/v3/projects/e18c722103c144378f8498bbd8c11f82/users/3c93baa0ed244642998a63289e775d6d/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:21.324409 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bf875-6119-4c68-8ee1-a3cd2a361be5 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:21.324903 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bf875-6119-4c68-8ee1-a3cd2a361be5 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:21.325932 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bf875-6119-4c68-8ee1-a3cd2a361be5 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:21.329048 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 869/1747] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jun 23 09:24:21 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:24:21.331233 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9e478-94a8-48c0-8f94-527422fef147 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:21.331743 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9e478-94a8-48c0-8f94-527422fef147 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:21.332179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9e478-94a8-48c0-8f94-527422fef147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:21.406865 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 879/1748] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jun 23 09:24:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:21.447377 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-27e9e478-94a8-48c0-8f94-527422fef147 None None] MFA Rules not processed for user `3c93baa0ed244642998a63289e775d6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:21.448895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-27e9e478-94a8-48c0-8f94-527422fef147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:21.499744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14309d10-cd19-44f6-9652-5f40fa9efba6 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:21.500047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14309d10-cd19-44f6-9652-5f40fa9efba6 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:21.500349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14309d10-cd19-44f6-9652-5f40fa9efba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:21.504408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:21.555839 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-14309d10-cd19-44f6-9652-5f40fa9efba6 None None] MFA Rules not processed for user `ff025cb706d243778468ae4570374690`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:21.564172 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-14309d10-cd19-44f6-9652-5f40fa9efba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:21.672112 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 870/1749] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:21.725498 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 880/1750] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jun 23 09:24:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:21.749226 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9e478-94a8-48c0-8f94-527422fef147 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:21.752491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-27e9e478-94a8-48c0-8f94-527422fef147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:21.859111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9e478-94a8-48c0-8f94-527422fef147 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:21.878831 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49850e-cf60-48fa-8b14-d76660c205a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:21.880155 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49850e-cf60-48fa-8b14-d76660c205a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:21.880446 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49850e-cf60-48fa-8b14-d76660c205a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:21.880728 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49850e-cf60-48fa-8b14-d76660c205a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:21.915885 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49850e-cf60-48fa-8b14-d76660c205a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:21.918556 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49850e-cf60-48fa-8b14-d76660c205a4 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:21.931998 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 871/1751] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:21.982232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14309d10-cd19-44f6-9652-5f40fa9efba6 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:21.985393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-14309d10-cd19-44f6-9652-5f40fa9efba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:22.004344 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14309d10-cd19-44f6-9652-5f40fa9efba6 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:22.013829 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f7697-1734-4ea1-a75a-ace63b636f5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:22.015959 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f7697-1734-4ea1-a75a-ace63b636f5a service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:22.015959 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f7697-1734-4ea1-a75a-ace63b636f5a service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:22.015959 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f7697-1734-4ea1-a75a-ace63b636f5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:22.037484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f7697-1734-4ea1-a75a-ace63b636f5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:22.037484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f7697-1734-4ea1-a75a-ace63b636f5a service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:22.062011 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49850e-cf60-48fa-8b14-d76660c205a4 service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:22.063523 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5f49850e-cf60-48fa-8b14-d76660c205a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:22.063981 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 881/1752] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:22.076114 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49850e-cf60-48fa-8b14-d76660c205a4 service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:22.080239 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526f998-bcb1-4768-9d31-10c816735773 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:22.083558 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526f998-bcb1-4768-9d31-10c816735773 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:22.083558 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526f998-bcb1-4768-9d31-10c816735773 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:22.083558 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526f998-bcb1-4768-9d31-10c816735773 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:22.190385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526f998-bcb1-4768-9d31-10c816735773 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:22.192078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526f998-bcb1-4768-9d31-10c816735773 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:22.197105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f7697-1734-4ea1-a75a-ace63b636f5a service nova] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:22.208013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 872/1753] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:22.225521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f7697-1734-4ea1-a75a-ace63b636f5a service nova] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:22.231093 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a36095-1d9c-4b58-b79a-b0f118ba2f28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:22.232450 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a36095-1d9c-4b58-b79a-b0f118ba2f28 service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:22.232742 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a36095-1d9c-4b58-b79a-b0f118ba2f28 service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:22.233021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a36095-1d9c-4b58-b79a-b0f118ba2f28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:22.244171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a36095-1d9c-4b58-b79a-b0f118ba2f28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:22.246257 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a36095-1d9c-4b58-b79a-b0f118ba2f28 service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:22.262034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 882/1754] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:24:22.325417 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526f998-bcb1-4768-9d31-10c816735773 service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:22.327979 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-8526f998-bcb1-4768-9d31-10c816735773 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:22.447409 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526f998-bcb1-4768-9d31-10c816735773 service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:22.452814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12bf180-1e2b-4ec4-9438-8af87c9a6b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:22.453715 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bf180-1e2b-4ec4-9438-8af87c9a6b94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:22.453999 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bf180-1e2b-4ec4-9438-8af87c9a6b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:22.454285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bf180-1e2b-4ec4-9438-8af87c9a6b94 admin admin] PATH_INFO: `/v3/users/58ec7836481d4b4ab65985d52e03d33f` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:22.455664 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12bf180-1e2b-4ec4-9438-8af87c9a6b94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58ec7836481d4b4ab65985d52e03d33f)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:22.457798 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12bf180-1e2b-4ec4-9438-8af87c9a6b94 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:22.734345 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f12bf180-1e2b-4ec4-9438-8af87c9a6b94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58ec7836481d4b4ab65985d52e03d33f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:22.741005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f12bf180-1e2b-4ec4-9438-8af87c9a6b94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58ec7836481d4b4ab65985d52e03d33f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:22.741661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f12bf180-1e2b-4ec4-9438-8af87c9a6b94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58ec7836481d4b4ab65985d52e03d33f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:22.742274 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f12bf180-1e2b-4ec4-9438-8af87c9a6b94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58ec7836481d4b4ab65985d52e03d33f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:22.754783 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f12bf180-1e2b-4ec4-9438-8af87c9a6b94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58ec7836481d4b4ab65985d52e03d33f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:22.875783 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 873/1755] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:24:22 2022] DELETE /identity/v3/users/58ec7836481d4b4ab65985d52e03d33f => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:22.882564 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ea28e-db9f-449e-abb2-691971e256e7 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:22.884442 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ea28e-db9f-449e-abb2-691971e256e7 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:22.884741 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ea28e-db9f-449e-abb2-691971e256e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:22.903671 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-f37ea28e-db9f-449e-abb2-691971e256e7 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:22.904930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-f37ea28e-db9f-449e-abb2-691971e256e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:22.963756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 883/1756] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:22.984987 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12bf180-1e2b-4ec4-9438-8af87c9a6b94 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:22.986284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-f12bf180-1e2b-4ec4-9438-8af87c9a6b94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:23.071734 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12bf180-1e2b-4ec4-9438-8af87c9a6b94 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:23.082868 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b0198-dcfb-4310-85e0-91bbc1f50aa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:23.083565 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b0198-dcfb-4310-85e0-91bbc1f50aa3 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:23.083854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b0198-dcfb-4310-85e0-91bbc1f50aa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:23.084134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b0198-dcfb-4310-85e0-91bbc1f50aa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:23.099761 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b0198-dcfb-4310-85e0-91bbc1f50aa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:23.101563 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b0198-dcfb-4310-85e0-91bbc1f50aa3 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:23.117401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 874/1757] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:24:23.612763 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:23.767801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ea28e-db9f-449e-abb2-691971e256e7 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:23.769209 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-f37ea28e-db9f-449e-abb2-691971e256e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:23.781498 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:23.785853 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ea28e-db9f-449e-abb2-691971e256e7 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:23.790696 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b9dfd-7c7f-4a70-b360-e687465d9166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:23.791613 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b9dfd-7c7f-4a70-b360-e687465d9166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:23.791869 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b9dfd-7c7f-4a70-b360-e687465d9166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:23.792247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b9dfd-7c7f-4a70-b360-e687465d9166 admin admin] PATH_INFO: `/v3/projects/7e77ff181ca6451d9039171e88e31f5f` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:23.793520 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b9dfd-7c7f-4a70-b360-e687465d9166 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e77ff181ca6451d9039171e88e31f5f)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:23.795460 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b9dfd-7c7f-4a70-b360-e687465d9166 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:23.897619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-a92b9dfd-7c7f-4a70-b360-e687465d9166 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e77ff181ca6451d9039171e88e31f5f', 'request_id': 'req-a92b9dfd-7c7f-4a70-b360-e687465d9166'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:23.904431 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-a92b9dfd-7c7f-4a70-b360-e687465d9166 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e77ff181ca6451d9039171e88e31f5f', 'request_id': 'req-a92b9dfd-7c7f-4a70-b360-e687465d9166'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:23.909689 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-a92b9dfd-7c7f-4a70-b360-e687465d9166 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e77ff181ca6451d9039171e88e31f5f', 'request_id': 'req-a92b9dfd-7c7f-4a70-b360-e687465d9166'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:23.993977 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-a92b9dfd-7c7f-4a70-b360-e687465d9166 admin admin] The token cache is being invalidate because project 7e77ff181ca6451d9039171e88e31f5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:24:23.994271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-a92b9dfd-7c7f-4a70-b360-e687465d9166 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:23.994895 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-a92b9dfd-7c7f-4a70-b360-e687465d9166 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:23.996629 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 884/1758] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:24:23 2022] DELETE /identity/v3/projects/7e77ff181ca6451d9039171e88e31f5f => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:24.001096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b0198-dcfb-4310-85e0-91bbc1f50aa3 service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:24.015608 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b0198-dcfb-4310-85e0-91bbc1f50aa3 service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:24.066251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e554b-3128-4cf6-8c50-8bec6e91225e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:24.067318 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e554b-3128-4cf6-8c50-8bec6e91225e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:24.067441 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e554b-3128-4cf6-8c50-8bec6e91225e admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:24.067723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e554b-3128-4cf6-8c50-8bec6e91225e admin admin] PATH_INFO: `/v3/users/fb44b8ba8c07431a9badfc4a7cd92a74` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:24.068995 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e554b-3128-4cf6-8c50-8bec6e91225e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb44b8ba8c07431a9badfc4a7cd92a74)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:24.070980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e554b-3128-4cf6-8c50-8bec6e91225e admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:24.306112 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-322e554b-3128-4cf6-8c50-8bec6e91225e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb44b8ba8c07431a9badfc4a7cd92a74'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:24.313344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-322e554b-3128-4cf6-8c50-8bec6e91225e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb44b8ba8c07431a9badfc4a7cd92a74'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:24.314085 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-322e554b-3128-4cf6-8c50-8bec6e91225e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb44b8ba8c07431a9badfc4a7cd92a74'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:24.314726 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-322e554b-3128-4cf6-8c50-8bec6e91225e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb44b8ba8c07431a9badfc4a7cd92a74'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:24.329724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-322e554b-3128-4cf6-8c50-8bec6e91225e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb44b8ba8c07431a9badfc4a7cd92a74'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:24.441720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 875/1759] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:24:24 2022] DELETE /identity/v3/users/fb44b8ba8c07431a9badfc4a7cd92a74 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:25.000231 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b9dfd-7c7f-4a70-b360-e687465d9166 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:25.002233 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-a92b9dfd-7c7f-4a70-b360-e687465d9166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:25.019808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b9dfd-7c7f-4a70-b360-e687465d9166 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:25.121350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2565284-624b-4059-8879-5c9a30055b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:25.123299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2565284-624b-4059-8879-5c9a30055b3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:25.123765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2565284-624b-4059-8879-5c9a30055b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:25.124145 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2565284-624b-4059-8879-5c9a30055b3a admin admin] PATH_INFO: `/v3/projects/cdc74866edd347b288deeeeb01e6f360` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:25.125669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2565284-624b-4059-8879-5c9a30055b3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdc74866edd347b288deeeeb01e6f360)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:25.127906 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2565284-624b-4059-8879-5c9a30055b3a admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:25.324335 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-a2565284-624b-4059-8879-5c9a30055b3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdc74866edd347b288deeeeb01e6f360', 'request_id': 'req-a2565284-624b-4059-8879-5c9a30055b3a'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:25.331148 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-a2565284-624b-4059-8879-5c9a30055b3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdc74866edd347b288deeeeb01e6f360', 'request_id': 'req-a2565284-624b-4059-8879-5c9a30055b3a'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:25.342097 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-a2565284-624b-4059-8879-5c9a30055b3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdc74866edd347b288deeeeb01e6f360', 'request_id': 'req-a2565284-624b-4059-8879-5c9a30055b3a'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:25.948072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-a2565284-624b-4059-8879-5c9a30055b3a admin admin] The token cache is being invalidate because project cdc74866edd347b288deeeeb01e6f360 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:24:25.948361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-a2565284-624b-4059-8879-5c9a30055b3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:25.949108 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-a2565284-624b-4059-8879-5c9a30055b3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:25.950681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 885/1760] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:24:24 2022] DELETE /identity/v3/projects/cdc74866edd347b288deeeeb01e6f360 => generated 0 bytes in 951 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:25.955785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14845410-8671-47d8-8049-7f45d2604bb1 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:25.956308 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14845410-8671-47d8-8049-7f45d2604bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:25.956407 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14845410-8671-47d8-8049-7f45d2604bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:25.979384 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-14845410-8671-47d8-8049-7f45d2604bb1 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:25.988753 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-14845410-8671-47d8-8049-7f45d2604bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:26.090760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 876/1761] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:24:26.092522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edec631-a5c4-4b2e-8d1c-fa882f71561c None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:26.093002 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edec631-a5c4-4b2e-8d1c-fa882f71561c None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:26.093092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edec631-a5c4-4b2e-8d1c-fa882f71561c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:26.113666 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-4edec631-a5c4-4b2e-8d1c-fa882f71561c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:26.115256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-4edec631-a5c4-4b2e-8d1c-fa882f71561c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:26.124516 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 886/1762] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:26.140709 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14845410-8671-47d8-8049-7f45d2604bb1 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:26.151946 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14845410-8671-47d8-8049-7f45d2604bb1 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:26.160309 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d394dcf-b70a-4af8-9681-2ff132d8f285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:26.161026 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d394dcf-b70a-4af8-9681-2ff132d8f285 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:26.161340 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d394dcf-b70a-4af8-9681-2ff132d8f285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:26.161599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d394dcf-b70a-4af8-9681-2ff132d8f285 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:26.162355 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d394dcf-b70a-4af8-9681-2ff132d8f285 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:24:26.162666 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d394dcf-b70a-4af8-9681-2ff132d8f285 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:26.164374 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d394dcf-b70a-4af8-9681-2ff132d8f285 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:26.179373 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 877/1763] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:24:26.186123 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edec631-a5c4-4b2e-8d1c-fa882f71561c None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:26.197325 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edec631-a5c4-4b2e-8d1c-fa882f71561c None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:26.203107 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25c6cf-8a4f-4e10-9c74-907198580a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:26.203875 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25c6cf-8a4f-4e10-9c74-907198580a8a admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:26.204230 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25c6cf-8a4f-4e10-9c74-907198580a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:26.204576 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25c6cf-8a4f-4e10-9c74-907198580a8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:26.205407 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25c6cf-8a4f-4e10-9c74-907198580a8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:26.207438 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25c6cf-8a4f-4e10-9c74-907198580a8a admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:26.517852 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 887/1764] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:24:26 2022] POST /identity/v3/projects => generated 381 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:24:26.521953 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d394dcf-b70a-4af8-9681-2ff132d8f285 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:26.534225 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d394dcf-b70a-4af8-9681-2ff132d8f285 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:26.602551 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:26.609489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13992a9a-72d7-44e6-a1f6-3cdfadd5c95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:26.610394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13992a9a-72d7-44e6-a1f6-3cdfadd5c95f admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:26.611686 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13992a9a-72d7-44e6-a1f6-3cdfadd5c95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:26.611976 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13992a9a-72d7-44e6-a1f6-3cdfadd5c95f admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:26.612693 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13992a9a-72d7-44e6-a1f6-3cdfadd5c95f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:26.614972 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13992a9a-72d7-44e6-a1f6-3cdfadd5c95f admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:26.741185 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 878/1765] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:24:26 2022] POST /identity/v3/users => generated 291 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:26.745207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25c6cf-8a4f-4e10-9c74-907198580a8a admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:26.770505 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25c6cf-8a4f-4e10-9c74-907198580a8a admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:26.781443 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9c597-7594-4b33-a1d5-b296d9c4463a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:26.782311 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9c597-7594-4b33-a1d5-b296d9c4463a admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:26.782748 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9c597-7594-4b33-a1d5-b296d9c4463a admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:26.783222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9c597-7594-4b33-a1d5-b296d9c4463a admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:26.784118 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9c597-7594-4b33-a1d5-b296d9c4463a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:26.788285 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9c597-7594-4b33-a1d5-b296d9c4463a admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:26.813223 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13992a9a-72d7-44e6-a1f6-3cdfadd5c95f admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:26.813900 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 888/1766] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:26.828857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13992a9a-72d7-44e6-a1f6-3cdfadd5c95f admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:26.846316 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9a09a-6b8f-4a98-b077-78270e5ebc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:26.857539 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9a09a-6b8f-4a98-b077-78270e5ebc71 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:26.857539 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9a09a-6b8f-4a98-b077-78270e5ebc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:26.857539 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9a09a-6b8f-4a98-b077-78270e5ebc71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:26.857539 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9a09a-6b8f-4a98-b077-78270e5ebc71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:26.857539 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9a09a-6b8f-4a98-b077-78270e5ebc71 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:26.873785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 879/1767] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:24:26.878461 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9c597-7594-4b33-a1d5-b296d9c4463a admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:26.892203 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9c597-7594-4b33-a1d5-b296d9c4463a admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:26.897198 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f0204-9feb-4d0c-8e55-3b3cf5175786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:26.898209 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f0204-9feb-4d0c-8e55-3b3cf5175786 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:26.898520 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f0204-9feb-4d0c-8e55-3b3cf5175786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:26.898958 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f0204-9feb-4d0c-8e55-3b3cf5175786 admin admin] PATH_INFO: `/v3/projects/e3f624338dbc488682e49a9b5000d063/users/de4e4fee335340029e69768a0ce4547f/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:26.915995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:26.930375 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f0204-9feb-4d0c-8e55-3b3cf5175786 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3f624338dbc488682e49a9b5000d063, user_id=de4e4fee335340029e69768a0ce4547f, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:26.933253 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f0204-9feb-4d0c-8e55-3b3cf5175786 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:27.066585 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fb2ca-08f0-4a2f-985c-4ddc03594b5e None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:27.067266 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fb2ca-08f0-4a2f-985c-4ddc03594b5e None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:27.067412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 889/1768] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:26 2022] PUT /identity/v3/projects/e3f624338dbc488682e49a9b5000d063/users/de4e4fee335340029e69768a0ce4547f/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:27.067520 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fb2ca-08f0-4a2f-985c-4ddc03594b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:27.119023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-d06fb2ca-08f0-4a2f-985c-4ddc03594b5e None None] MFA Rules not processed for user `de4e4fee335340029e69768a0ce4547f`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:27.120444 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d06fb2ca-08f0-4a2f-985c-4ddc03594b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:27.207492 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 880/1769] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:24:27.212405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f0204-9feb-4d0c-8e55-3b3cf5175786 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:27.302962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f0204-9feb-4d0c-8e55-3b3cf5175786 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:27.308545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b10c1-7339-49fb-806c-62764da9fbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:27.309491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b10c1-7339-49fb-806c-62764da9fbbf admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:27.309895 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b10c1-7339-49fb-806c-62764da9fbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:27.310268 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b10c1-7339-49fb-806c-62764da9fbbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:27.311119 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b10c1-7339-49fb-806c-62764da9fbbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:27.313457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b10c1-7339-49fb-806c-62764da9fbbf admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:27.503517 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 890/1770] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:24:27 2022] POST /identity/v3/projects => generated 383 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:27.507446 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fb2ca-08f0-4a2f-985c-4ddc03594b5e None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:27.525289 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fb2ca-08f0-4a2f-985c-4ddc03594b5e None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:27.611132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad22b8-b508-4ab0-a928-7596b5f753f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:27.611975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad22b8-b508-4ab0-a928-7596b5f753f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:27.612342 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad22b8-b508-4ab0-a928-7596b5f753f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:27.612885 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad22b8-b508-4ab0-a928-7596b5f753f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:27.617124 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad22b8-b508-4ab0-a928-7596b5f753f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:27.617124 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad22b8-b508-4ab0-a928-7596b5f753f1 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:27.744501 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 881/1771] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:24:27 2022] POST /identity/v3/users => generated 298 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:27.749331 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b10c1-7339-49fb-806c-62764da9fbbf admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:27.768563 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b10c1-7339-49fb-806c-62764da9fbbf admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:27.773820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3f3f5-b90a-4000-b6c2-a716333d7a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:27.774557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3f3f5-b90a-4000-b6c2-a716333d7a58 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:27.774848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3f3f5-b90a-4000-b6c2-a716333d7a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:27.775183 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3f3f5-b90a-4000-b6c2-a716333d7a58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:27.775921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3f3f5-b90a-4000-b6c2-a716333d7a58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:27.777661 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3f3f5-b90a-4000-b6c2-a716333d7a58 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:27.799934 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 891/1772] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:27.804170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad22b8-b508-4ab0-a928-7596b5f753f1 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:27.819687 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad22b8-b508-4ab0-a928-7596b5f753f1 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:27.824968 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3018dc-0cb0-42ee-9c04-95b62d6e3e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:27.826002 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3018dc-0cb0-42ee-9c04-95b62d6e3e3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:27.826392 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3018dc-0cb0-42ee-9c04-95b62d6e3e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:27.826772 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3018dc-0cb0-42ee-9c04-95b62d6e3e3b admin admin] PATH_INFO: `/v3/projects/913be6d3b78149289001b96f667e6e2d/users/9cef83396cd44f9c90a63ea0846fb42a/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:27.849561 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3018dc-0cb0-42ee-9c04-95b62d6e3e3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=913be6d3b78149289001b96f667e6e2d, user_id=9cef83396cd44f9c90a63ea0846fb42a, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:27.852431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3018dc-0cb0-42ee-9c04-95b62d6e3e3b admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:27.979220 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 882/1773] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:27 2022] PUT /identity/v3/projects/913be6d3b78149289001b96f667e6e2d/users/9cef83396cd44f9c90a63ea0846fb42a/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:27.983912 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3f3f5-b90a-4000-b6c2-a716333d7a58 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:28.008781 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3f3f5-b90a-4000-b6c2-a716333d7a58 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:28.095484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600173e2-db89-4ed7-a038-0dfdad3f56f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:28.096358 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600173e2-db89-4ed7-a038-0dfdad3f56f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:28.096940 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600173e2-db89-4ed7-a038-0dfdad3f56f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:28.098297 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600173e2-db89-4ed7-a038-0dfdad3f56f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:28.098297 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600173e2-db89-4ed7-a038-0dfdad3f56f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:28.099869 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600173e2-db89-4ed7-a038-0dfdad3f56f2 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:28.113593 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 892/1774] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:24:28.121973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3018dc-0cb0-42ee-9c04-95b62d6e3e3b admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:28.144872 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3018dc-0cb0-42ee-9c04-95b62d6e3e3b admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:28.151658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411e698-7ed8-4d95-893b-510f1bc4e9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:28.152943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411e698-7ed8-4d95-893b-510f1bc4e9f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:28.153873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411e698-7ed8-4d95-893b-510f1bc4e9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:28.154251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411e698-7ed8-4d95-893b-510f1bc4e9f8 admin admin] PATH_INFO: `/v3/projects/913be6d3b78149289001b96f667e6e2d/users/9cef83396cd44f9c90a63ea0846fb42a/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:28.156645 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411e698-7ed8-4d95-893b-510f1bc4e9f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=913be6d3b78149289001b96f667e6e2d, user_id=9cef83396cd44f9c90a63ea0846fb42a, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:28.159604 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411e698-7ed8-4d95-893b-510f1bc4e9f8 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:28.176022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 883/1775] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:28 2022] PUT /identity/v3/projects/913be6d3b78149289001b96f667e6e2d/users/9cef83396cd44f9c90a63ea0846fb42a/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:28.187884 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60724506-f9f9-4fad-aef2-0c8f54a63492 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:28.188313 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60724506-f9f9-4fad-aef2-0c8f54a63492 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:28.188785 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60724506-f9f9-4fad-aef2-0c8f54a63492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:28.231650 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-60724506-f9f9-4fad-aef2-0c8f54a63492 None None] MFA Rules not processed for user `9cef83396cd44f9c90a63ea0846fb42a`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:28.233449 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-60724506-f9f9-4fad-aef2-0c8f54a63492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:28.321660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 893/1776] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:28.363866 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411e698-7ed8-4d95-893b-510f1bc4e9f8 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:28.365607 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-e411e698-7ed8-4d95-893b-510f1bc4e9f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:28.445126 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411e698-7ed8-4d95-893b-510f1bc4e9f8 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:28.448956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a2932-edb2-461a-97a4-fb4740bb7b26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:28.449617 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a2932-edb2-461a-97a4-fb4740bb7b26 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:28.449946 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a2932-edb2-461a-97a4-fb4740bb7b26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:28.450219 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a2932-edb2-461a-97a4-fb4740bb7b26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:28.460011 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a2932-edb2-461a-97a4-fb4740bb7b26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:28.461545 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a2932-edb2-461a-97a4-fb4740bb7b26 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:28.472739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 884/1777] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:28.784024 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:28.901137 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60724506-f9f9-4fad-aef2-0c8f54a63492 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:28.903594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-60724506-f9f9-4fad-aef2-0c8f54a63492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:28.919620 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60724506-f9f9-4fad-aef2-0c8f54a63492 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:28.926867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f726b7-5abc-42e8-aed6-7fa55aa95a4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:28.928379 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f726b7-5abc-42e8-aed6-7fa55aa95a4a service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:28.928731 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f726b7-5abc-42e8-aed6-7fa55aa95a4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:28.929017 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f726b7-5abc-42e8-aed6-7fa55aa95a4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:28.988453 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f726b7-5abc-42e8-aed6-7fa55aa95a4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:28.989898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f726b7-5abc-42e8-aed6-7fa55aa95a4a service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:29.000455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 894/1778] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:29.758060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a2932-edb2-461a-97a4-fb4740bb7b26 service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:29.825883 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a2932-edb2-461a-97a4-fb4740bb7b26 service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:29.830386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb61bc-327f-4e3d-9cf5-c3f9150b8398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:29.831298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb61bc-327f-4e3d-9cf5-c3f9150b8398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:29.831587 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb61bc-327f-4e3d-9cf5-c3f9150b8398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:29.831865 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb61bc-327f-4e3d-9cf5-c3f9150b8398 admin admin] PATH_INFO: `/v3/users/de4e4fee335340029e69768a0ce4547f` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:29.833208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb61bc-327f-4e3d-9cf5-c3f9150b8398 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de4e4fee335340029e69768a0ce4547f)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:29.835071 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb61bc-327f-4e3d-9cf5-c3f9150b8398 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:30.107761 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-28cb61bc-327f-4e3d-9cf5-c3f9150b8398 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de4e4fee335340029e69768a0ce4547f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:30.114778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-28cb61bc-327f-4e3d-9cf5-c3f9150b8398 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de4e4fee335340029e69768a0ce4547f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:30.115491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-28cb61bc-327f-4e3d-9cf5-c3f9150b8398 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de4e4fee335340029e69768a0ce4547f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:30.116102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-28cb61bc-327f-4e3d-9cf5-c3f9150b8398 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de4e4fee335340029e69768a0ce4547f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:30.129085 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-28cb61bc-327f-4e3d-9cf5-c3f9150b8398 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de4e4fee335340029e69768a0ce4547f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:30.286895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 885/1779] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:24:29 2022] DELETE /identity/v3/users/de4e4fee335340029e69768a0ce4547f => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:30.291647 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107d3df-d47f-4ea1-aab2-65f77425787a None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:30.292147 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107d3df-d47f-4ea1-aab2-65f77425787a None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:30.292280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107d3df-d47f-4ea1-aab2-65f77425787a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:30.311834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-a107d3df-d47f-4ea1-aab2-65f77425787a None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:30.313138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-a107d3df-d47f-4ea1-aab2-65f77425787a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:30.377371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 895/1780] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:24:30.378507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:30.386114 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb61bc-327f-4e3d-9cf5-c3f9150b8398 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:30.387642 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-28cb61bc-327f-4e3d-9cf5-c3f9150b8398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:30.443413 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb61bc-327f-4e3d-9cf5-c3f9150b8398 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:30.447157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf4305-75d4-49de-ac5b-ed3074adb3be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:30.447802 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf4305-75d4-49de-ac5b-ed3074adb3be service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:30.448087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf4305-75d4-49de-ac5b-ed3074adb3be service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:30.448366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf4305-75d4-49de-ac5b-ed3074adb3be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:30.459576 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf4305-75d4-49de-ac5b-ed3074adb3be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:30.461014 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf4305-75d4-49de-ac5b-ed3074adb3be service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:30.470905 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 886/1781] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:31.008219 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107d3df-d47f-4ea1-aab2-65f77425787a None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:31.014214 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-a107d3df-d47f-4ea1-aab2-65f77425787a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:31.023098 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107d3df-d47f-4ea1-aab2-65f77425787a None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:31.029295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4714e-031f-4341-a7c0-231e162b1425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:31.030161 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4714e-031f-4341-a7c0-231e162b1425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:31.030449 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4714e-031f-4341-a7c0-231e162b1425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:31.030731 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4714e-031f-4341-a7c0-231e162b1425 admin admin] PATH_INFO: `/v3/projects/e3f624338dbc488682e49a9b5000d063` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:31.032228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4714e-031f-4341-a7c0-231e162b1425 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3f624338dbc488682e49a9b5000d063)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:31.034277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4714e-031f-4341-a7c0-231e162b1425 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:31.174388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b9d4714e-031f-4341-a7c0-231e162b1425 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3f624338dbc488682e49a9b5000d063', 'request_id': 'req-b9d4714e-031f-4341-a7c0-231e162b1425'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:31.180826 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b9d4714e-031f-4341-a7c0-231e162b1425 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3f624338dbc488682e49a9b5000d063', 'request_id': 'req-b9d4714e-031f-4341-a7c0-231e162b1425'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:31.186407 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b9d4714e-031f-4341-a7c0-231e162b1425 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3f624338dbc488682e49a9b5000d063', 'request_id': 'req-b9d4714e-031f-4341-a7c0-231e162b1425'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:31.190036 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:31.326429 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b9d4714e-031f-4341-a7c0-231e162b1425 admin admin] The token cache is being invalidate because project e3f624338dbc488682e49a9b5000d063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:24:31.327037 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b9d4714e-031f-4341-a7c0-231e162b1425 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:31.330003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b9d4714e-031f-4341-a7c0-231e162b1425 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:31.332598 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 896/1782] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:24:31 2022] DELETE /identity/v3/projects/e3f624338dbc488682e49a9b5000d063 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:31.336847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf4305-75d4-49de-ac5b-ed3074adb3be service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:31.347165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf4305-75d4-49de-ac5b-ed3074adb3be service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:31.401597 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f09c49-7acd-429d-94d6-048449ed89ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:31.402495 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f09c49-7acd-429d-94d6-048449ed89ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:31.402777 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f09c49-7acd-429d-94d6-048449ed89ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:31.403658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f09c49-7acd-429d-94d6-048449ed89ab admin admin] PATH_INFO: `/v3/users/9cef83396cd44f9c90a63ea0846fb42a` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:31.404912 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f09c49-7acd-429d-94d6-048449ed89ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cef83396cd44f9c90a63ea0846fb42a)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:31.406856 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f09c49-7acd-429d-94d6-048449ed89ab admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:31.690729 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-b9f09c49-7acd-429d-94d6-048449ed89ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cef83396cd44f9c90a63ea0846fb42a'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:31.690729 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-b9f09c49-7acd-429d-94d6-048449ed89ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cef83396cd44f9c90a63ea0846fb42a'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:31.691385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-b9f09c49-7acd-429d-94d6-048449ed89ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cef83396cd44f9c90a63ea0846fb42a'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:31.692023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-b9f09c49-7acd-429d-94d6-048449ed89ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cef83396cd44f9c90a63ea0846fb42a'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:31.704548 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-b9f09c49-7acd-429d-94d6-048449ed89ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cef83396cd44f9c90a63ea0846fb42a'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:31.840893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 887/1783] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:24:31 2022] DELETE /identity/v3/users/9cef83396cd44f9c90a63ea0846fb42a => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:31.961387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:32.490386 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4714e-031f-4341-a7c0-231e162b1425 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:32.492729 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b9d4714e-031f-4341-a7c0-231e162b1425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:32.508277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4714e-031f-4341-a7c0-231e162b1425 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:32.583823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5888dfc-5106-44f3-9684-ae24eb232b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:32.585295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5888dfc-5106-44f3-9684-ae24eb232b38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:32.586573 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5888dfc-5106-44f3-9684-ae24eb232b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:32.587203 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5888dfc-5106-44f3-9684-ae24eb232b38 admin admin] PATH_INFO: `/v3/projects/913be6d3b78149289001b96f667e6e2d` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:32.589393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5888dfc-5106-44f3-9684-ae24eb232b38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=913be6d3b78149289001b96f667e6e2d)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:32.592304 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5888dfc-5106-44f3-9684-ae24eb232b38 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:32.746487 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:32.758819 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c5888dfc-5106-44f3-9684-ae24eb232b38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '913be6d3b78149289001b96f667e6e2d', 'request_id': 'req-c5888dfc-5106-44f3-9684-ae24eb232b38'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:32.766004 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c5888dfc-5106-44f3-9684-ae24eb232b38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '913be6d3b78149289001b96f667e6e2d', 'request_id': 'req-c5888dfc-5106-44f3-9684-ae24eb232b38'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:32.772590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c5888dfc-5106-44f3-9684-ae24eb232b38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '913be6d3b78149289001b96f667e6e2d', 'request_id': 'req-c5888dfc-5106-44f3-9684-ae24eb232b38'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:33.043778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:33.050175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c5888dfc-5106-44f3-9684-ae24eb232b38 admin admin] The token cache is being invalidate because project 913be6d3b78149289001b96f667e6e2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:24:33.050611 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c5888dfc-5106-44f3-9684-ae24eb232b38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:33.051472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c5888dfc-5106-44f3-9684-ae24eb232b38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:33.054330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 897/1784] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:24:32 2022] DELETE /identity/v3/projects/913be6d3b78149289001b96f667e6e2d => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:33.060333 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d0767-fdab-4e45-b145-88ed36d120d7 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:33.061094 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d0767-fdab-4e45-b145-88ed36d120d7 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:33.061510 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d0767-fdab-4e45-b145-88ed36d120d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:33.084847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-5b3d0767-fdab-4e45-b145-88ed36d120d7 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:33.086307 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5b3d0767-fdab-4e45-b145-88ed36d120d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:33.205851 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 888/1785] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:33.216002 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e45021-0eaa-49e6-8182-ac8904e98b37 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:33.216401 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e45021-0eaa-49e6-8182-ac8904e98b37 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:33.217135 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e45021-0eaa-49e6-8182-ac8904e98b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:33.260560 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-b1e45021-0eaa-49e6-8182-ac8904e98b37 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:33.262060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b1e45021-0eaa-49e6-8182-ac8904e98b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:33.310819 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 898/1786] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:33.315387 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d0767-fdab-4e45-b145-88ed36d120d7 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:33.330688 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d0767-fdab-4e45-b145-88ed36d120d7 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:33.348407 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15f4c9-df2f-4d30-b3f9-75693ff76d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:33.348407 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15f4c9-df2f-4d30-b3f9-75693ff76d17 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:33.348407 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15f4c9-df2f-4d30-b3f9-75693ff76d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:33.348689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15f4c9-df2f-4d30-b3f9-75693ff76d17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:33.350064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15f4c9-df2f-4d30-b3f9-75693ff76d17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:24:33.350383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15f4c9-df2f-4d30-b3f9-75693ff76d17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:33.352076 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15f4c9-df2f-4d30-b3f9-75693ff76d17 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:33.367736 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 889/1787] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:24:33.373595 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e45021-0eaa-49e6-8182-ac8904e98b37 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:33.390428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e45021-0eaa-49e6-8182-ac8904e98b37 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:33.404648 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bf258-082f-4ac6-bea9-066fb28ac53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:33.406009 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bf258-082f-4ac6-bea9-066fb28ac53e admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:33.406503 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bf258-082f-4ac6-bea9-066fb28ac53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:33.407099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bf258-082f-4ac6-bea9-066fb28ac53e admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:33.408064 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8bf258-082f-4ac6-bea9-066fb28ac53e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:33.410348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8bf258-082f-4ac6-bea9-066fb28ac53e admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:33.476833 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15f4c9-df2f-4d30-b3f9-75693ff76d17 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:33.478858 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-8e15f4c9-df2f-4d30-b3f9-75693ff76d17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:33.496087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15f4c9-df2f-4d30-b3f9-75693ff76d17 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:33.510779 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda6be2-6739-4790-9be7-1268e6eeeff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:33.511783 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda6be2-6739-4790-9be7-1268e6eeeff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:33.524853 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda6be2-6739-4790-9be7-1268e6eeeff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:33.531917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda6be2-6739-4790-9be7-1268e6eeeff0 admin admin] PATH_INFO: `/v3/users/bbd74757c4fe4e6ebb1f3845d9ef11eb` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:33.531917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda6be2-6739-4790-9be7-1268e6eeeff0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbd74757c4fe4e6ebb1f3845d9ef11eb)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:33.533553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda6be2-6739-4790-9be7-1268e6eeeff0 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:33.591942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 899/1788] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:24:33 2022] POST /identity/v3/projects => generated 371 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:33.592872 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bf258-082f-4ac6-bea9-066fb28ac53e admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:33.655870 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bf258-082f-4ac6-bea9-066fb28ac53e admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:33.660617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbed52a-bfab-49dd-a8b9-5a10e5f261a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:33.661291 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbed52a-bfab-49dd-a8b9-5a10e5f261a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:33.661584 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbed52a-bfab-49dd-a8b9-5a10e5f261a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:33.661959 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbed52a-bfab-49dd-a8b9-5a10e5f261a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:33.662739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbed52a-bfab-49dd-a8b9-5a10e5f261a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:33.664742 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbed52a-bfab-49dd-a8b9-5a10e5f261a4 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:33.838632 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 900/1789] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:24:33 2022] POST /identity/v3/users => generated 286 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:24:33.843038 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbed52a-bfab-49dd-a8b9-5a10e5f261a4 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:33.855153 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbed52a-bfab-49dd-a8b9-5a10e5f261a4 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:33.860925 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d054b-3fa1-4191-9c1f-a06d6b779447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:33.861583 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d054b-3fa1-4191-9c1f-a06d6b779447 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:33.861934 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d054b-3fa1-4191-9c1f-a06d6b779447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:33.862156 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d054b-3fa1-4191-9c1f-a06d6b779447 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:33.862885 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d054b-3fa1-4191-9c1f-a06d6b779447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:33.864505 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d054b-3fa1-4191-9c1f-a06d6b779447 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:33.874267 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 901/1790] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:24:33.877913 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d054b-3fa1-4191-9c1f-a06d6b779447 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:33.887371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d054b-3fa1-4191-9c1f-a06d6b779447 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:33.891937 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48ef04-4b7f-45d4-b4a1-319ce727a0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:33.892565 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48ef04-4b7f-45d4-b4a1-319ce727a0d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:33.892918 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48ef04-4b7f-45d4-b4a1-319ce727a0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:33.893168 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48ef04-4b7f-45d4-b4a1-319ce727a0d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:33.893888 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc48ef04-4b7f-45d4-b4a1-319ce727a0d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:33.895500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc48ef04-4b7f-45d4-b4a1-319ce727a0d4 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:33.906391 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 902/1791] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:24:33.910151 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48ef04-4b7f-45d4-b4a1-319ce727a0d4 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:33.919853 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48ef04-4b7f-45d4-b4a1-319ce727a0d4 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:33.924429 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80349713-bbab-42ad-aad6-bc672a19bee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:33.925269 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80349713-bbab-42ad-aad6-bc672a19bee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:33.925554 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80349713-bbab-42ad-aad6-bc672a19bee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:33.925834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80349713-bbab-42ad-aad6-bc672a19bee2 admin admin] PATH_INFO: `/v3/projects/5a7afaa88f93446884e96bb39142d3b4/users/7e9ecac5434e4d40806f57d53256f7fa/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:33.940563 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80349713-bbab-42ad-aad6-bc672a19bee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a7afaa88f93446884e96bb39142d3b4, user_id=7e9ecac5434e4d40806f57d53256f7fa, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:33.943236 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80349713-bbab-42ad-aad6-bc672a19bee2 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:34.003095 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4fda6be2-6739-4790-9be7-1268e6eeeff0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbd74757c4fe4e6ebb1f3845d9ef11eb'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:34.009907 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4fda6be2-6739-4790-9be7-1268e6eeeff0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbd74757c4fe4e6ebb1f3845d9ef11eb'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:34.010587 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4fda6be2-6739-4790-9be7-1268e6eeeff0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbd74757c4fe4e6ebb1f3845d9ef11eb'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:34.011255 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4fda6be2-6739-4790-9be7-1268e6eeeff0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbd74757c4fe4e6ebb1f3845d9ef11eb'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:34.022863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4fda6be2-6739-4790-9be7-1268e6eeeff0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbd74757c4fe4e6ebb1f3845d9ef11eb'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:34.223954 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 903/1792] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:33 2022] PUT /identity/v3/projects/5a7afaa88f93446884e96bb39142d3b4/users/7e9ecac5434e4d40806f57d53256f7fa/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:24:34.228713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6028b0-10fd-4c0c-995e-be08f9586b3d None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:34.229013 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6028b0-10fd-4c0c-995e-be08f9586b3d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:34.229296 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6028b0-10fd-4c0c-995e-be08f9586b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:34.245910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-de6028b0-10fd-4c0c-995e-be08f9586b3d None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:34.247225 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-de6028b0-10fd-4c0c-995e-be08f9586b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:34.288198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 890/1793] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:24:33 2022] DELETE /identity/v3/users/bbd74757c4fe4e6ebb1f3845d9ef11eb => generated 0 bytes in 812 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:34.292673 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c3c69-5e75-4a9a-bdea-6c7f0e3cd370 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:34.293024 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c3c69-5e75-4a9a-bdea-6c7f0e3cd370 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:34.293310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c3c69-5e75-4a9a-bdea-6c7f0e3cd370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:34.309786 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 904/1794] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:24:34.311590 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-0e1c3c69-5e75-4a9a-bdea-6c7f0e3cd370 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:34.312772 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-0e1c3c69-5e75-4a9a-bdea-6c7f0e3cd370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:34.316164 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6028b0-10fd-4c0c-995e-be08f9586b3d None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:34.317524 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-de6028b0-10fd-4c0c-995e-be08f9586b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:34.379915 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6028b0-10fd-4c0c-995e-be08f9586b3d None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:34.380711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 891/1795] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:24:34.391076 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c3c69-5e75-4a9a-bdea-6c7f0e3cd370 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:34.392543 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-0e1c3c69-5e75-4a9a-bdea-6c7f0e3cd370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:34.432990 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952fb8d2-90fc-4638-9fd8-d40ac067ae2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:34.434071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fb8d2-90fc-4638-9fd8-d40ac067ae2a service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:34.434071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fb8d2-90fc-4638-9fd8-d40ac067ae2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:34.435206 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fb8d2-90fc-4638-9fd8-d40ac067ae2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:34.446321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952fb8d2-90fc-4638-9fd8-d40ac067ae2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:34.448107 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952fb8d2-90fc-4638-9fd8-d40ac067ae2a service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:34.453345 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c3c69-5e75-4a9a-bdea-6c7f0e3cd370 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:34.459602 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13746c8e-6be8-41aa-a428-27403079b7eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:34.460270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13746c8e-6be8-41aa-a428-27403079b7eb service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:34.460556 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13746c8e-6be8-41aa-a428-27403079b7eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:34.460971 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13746c8e-6be8-41aa-a428-27403079b7eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:34.461426 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 905/1796] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:24:34.472051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13746c8e-6be8-41aa-a428-27403079b7eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:34.473502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13746c8e-6be8-41aa-a428-27403079b7eb service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:34.484031 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 892/1797] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:24:34.822637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952fb8d2-90fc-4638-9fd8-d40ac067ae2a service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:34.824120 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-952fb8d2-90fc-4638-9fd8-d40ac067ae2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:34.838229 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952fb8d2-90fc-4638-9fd8-d40ac067ae2a service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:34.843520 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b033b-e595-4baa-b3b6-c539d1536fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:34.844409 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b033b-e595-4baa-b3b6-c539d1536fa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:34.844700 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b033b-e595-4baa-b3b6-c539d1536fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:34.844983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b033b-e595-4baa-b3b6-c539d1536fa0 admin admin] PATH_INFO: `/v3/users/368d1e8cf88943b3a05729daeff4cb8e` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:34.846372 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b033b-e595-4baa-b3b6-c539d1536fa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=368d1e8cf88943b3a05729daeff4cb8e)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:34.848156 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b033b-e595-4baa-b3b6-c539d1536fa0 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:35.283734 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-545b033b-e595-4baa-b3b6-c539d1536fa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '368d1e8cf88943b3a05729daeff4cb8e'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:35.285523 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13746c8e-6be8-41aa-a428-27403079b7eb service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:35.286826 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-13746c8e-6be8-41aa-a428-27403079b7eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:35.301847 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-545b033b-e595-4baa-b3b6-c539d1536fa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '368d1e8cf88943b3a05729daeff4cb8e'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:35.301960 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13746c8e-6be8-41aa-a428-27403079b7eb service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:35.302388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-545b033b-e595-4baa-b3b6-c539d1536fa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '368d1e8cf88943b3a05729daeff4cb8e'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:35.304897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-545b033b-e595-4baa-b3b6-c539d1536fa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '368d1e8cf88943b3a05729daeff4cb8e'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:35.308081 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895aeeb-7eac-4f1e-9775-13e77924cae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:35.308973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895aeeb-7eac-4f1e-9775-13e77924cae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:35.309982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895aeeb-7eac-4f1e-9775-13e77924cae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:35.313619 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895aeeb-7eac-4f1e-9775-13e77924cae3 admin admin] PATH_INFO: `/v3/projects/ced9fa9aa54d42dda557bb17d14fe971` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:35.317960 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895aeeb-7eac-4f1e-9775-13e77924cae3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ced9fa9aa54d42dda557bb17d14fe971)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:35.322091 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895aeeb-7eac-4f1e-9775-13e77924cae3 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:35.333367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-545b033b-e595-4baa-b3b6-c539d1536fa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '368d1e8cf88943b3a05729daeff4cb8e'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:35.489053 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 906/1798] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:24:34 2022] DELETE /identity/v3/users/368d1e8cf88943b3a05729daeff4cb8e => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:35.494428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ffda5-2537-44d5-8fbb-8ba1a1631c96 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:35.495040 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ffda5-2537-44d5-8fbb-8ba1a1631c96 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:35.495499 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ffda5-2537-44d5-8fbb-8ba1a1631c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:35.534215 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-9e5ffda5-2537-44d5-8fbb-8ba1a1631c96 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:35.535782 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-9e5ffda5-2537-44d5-8fbb-8ba1a1631c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:35.596122 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-9895aeeb-7eac-4f1e-9775-13e77924cae3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ced9fa9aa54d42dda557bb17d14fe971', 'request_id': 'req-9895aeeb-7eac-4f1e-9775-13e77924cae3'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:35.626825 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-9895aeeb-7eac-4f1e-9775-13e77924cae3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ced9fa9aa54d42dda557bb17d14fe971', 'request_id': 'req-9895aeeb-7eac-4f1e-9775-13e77924cae3'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:35.666476 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 907/1799] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:24:35.716523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ffda5-2537-44d5-8fbb-8ba1a1631c96 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:35.718533 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-9e5ffda5-2537-44d5-8fbb-8ba1a1631c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:35.760496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-9895aeeb-7eac-4f1e-9775-13e77924cae3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ced9fa9aa54d42dda557bb17d14fe971', 'request_id': 'req-9895aeeb-7eac-4f1e-9775-13e77924cae3'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:35.774986 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-9895aeeb-7eac-4f1e-9775-13e77924cae3 admin admin] The token cache is being invalidate because project ced9fa9aa54d42dda557bb17d14fe971 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:24:35.775851 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-9895aeeb-7eac-4f1e-9775-13e77924cae3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:35.777606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-9895aeeb-7eac-4f1e-9775-13e77924cae3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:35.780047 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 893/1800] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:24:35 2022] DELETE /identity/v3/projects/ced9fa9aa54d42dda557bb17d14fe971 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:35.784540 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7747a-d5a1-4450-bbd8-efb715799056 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:35.785031 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7747a-d5a1-4450-bbd8-efb715799056 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:35.785493 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7747a-d5a1-4450-bbd8-efb715799056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:35.803117 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:35.830681 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-95b7747a-d5a1-4450-bbd8-efb715799056 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:35.832036 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-95b7747a-d5a1-4450-bbd8-efb715799056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:35.838378 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ffda5-2537-44d5-8fbb-8ba1a1631c96 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:35.959379 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 894/1801] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:24:35.967809 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a10f2e-1b3f-419d-9194-e651810f43cc None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:35.968202 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a10f2e-1b3f-419d-9194-e651810f43cc None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:35.968584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a10f2e-1b3f-419d-9194-e651810f43cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:35.971330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cb6ab-0980-4797-b4ad-4dc419919d30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:35.973016 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cb6ab-0980-4797-b4ad-4dc419919d30 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:35.973608 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cb6ab-0980-4797-b4ad-4dc419919d30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:35.974148 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cb6ab-0980-4797-b4ad-4dc419919d30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:35.991625 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-8c8cb6ab-0980-4797-b4ad-4dc419919d30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:36.005340 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-47a10f2e-1b3f-419d-9194-e651810f43cc None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:36.007104 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-47a10f2e-1b3f-419d-9194-e651810f43cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:36.019996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cb6ab-0980-4797-b4ad-4dc419919d30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:36.021864 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cb6ab-0980-4797-b4ad-4dc419919d30 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:36.036426 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 908/1802] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:36.045946 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 895/1803] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:24:36.050833 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cb6ab-0980-4797-b4ad-4dc419919d30 service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:36.066801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cb6ab-0980-4797-b4ad-4dc419919d30 service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:36.074804 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77e259-5f28-4cd8-9adf-4f9220368925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:36.077662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77e259-5f28-4cd8-9adf-4f9220368925 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:36.078411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77e259-5f28-4cd8-9adf-4f9220368925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:36.078802 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77e259-5f28-4cd8-9adf-4f9220368925 admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:36.080459 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77e259-5f28-4cd8-9adf-4f9220368925 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:24:36.081634 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77e259-5f28-4cd8-9adf-4f9220368925 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:36.083706 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77e259-5f28-4cd8-9adf-4f9220368925 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:36.107850 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 909/1804] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:24:36.112678 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a10f2e-1b3f-419d-9194-e651810f43cc None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:36.126589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a10f2e-1b3f-419d-9194-e651810f43cc None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:36.133533 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfd2e4-cded-4a36-a506-1a6bb1fa7a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:36.134438 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfd2e4-cded-4a36-a506-1a6bb1fa7a52 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:36.134553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfd2e4-cded-4a36-a506-1a6bb1fa7a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:36.135510 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfd2e4-cded-4a36-a506-1a6bb1fa7a52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:36.136245 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfd2e4-cded-4a36-a506-1a6bb1fa7a52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:36.138265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfd2e4-cded-4a36-a506-1a6bb1fa7a52 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:36.371233 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 896/1805] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:24:36 2022] POST /identity/v3/projects => generated 385 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:36.375500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77e259-5f28-4cd8-9adf-4f9220368925 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:36.386045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77e259-5f28-4cd8-9adf-4f9220368925 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:36.453949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367738f2-df45-425e-9b8f-160152ebaf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:36.454671 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367738f2-df45-425e-9b8f-160152ebaf7c admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:36.454992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367738f2-df45-425e-9b8f-160152ebaf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:36.455276 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367738f2-df45-425e-9b8f-160152ebaf7c admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:36.456204 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367738f2-df45-425e-9b8f-160152ebaf7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:36.458212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367738f2-df45-425e-9b8f-160152ebaf7c admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:36.584461 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 910/1806] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:24:36 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:36.588409 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfd2e4-cded-4a36-a506-1a6bb1fa7a52 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:36.599911 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfd2e4-cded-4a36-a506-1a6bb1fa7a52 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:36.618548 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb45748-0b01-4052-bd05-2b428f544969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:36.619310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb45748-0b01-4052-bd05-2b428f544969 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:36.619601 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb45748-0b01-4052-bd05-2b428f544969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:36.619880 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb45748-0b01-4052-bd05-2b428f544969 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:36.620988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb45748-0b01-4052-bd05-2b428f544969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:36.623740 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb45748-0b01-4052-bd05-2b428f544969 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:36.640254 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 897/1807] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:36.641173 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367738f2-df45-425e-9b8f-160152ebaf7c admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:36.654111 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367738f2-df45-425e-9b8f-160152ebaf7c admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:36.659800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c2fd1-5db3-45a6-9e70-01fcb220b0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:36.660563 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c2fd1-5db3-45a6-9e70-01fcb220b0c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:36.660919 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c2fd1-5db3-45a6-9e70-01fcb220b0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:36.661404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c2fd1-5db3-45a6-9e70-01fcb220b0c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:36.662559 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c2fd1-5db3-45a6-9e70-01fcb220b0c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:36.664490 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c2fd1-5db3-45a6-9e70-01fcb220b0c8 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:36.677978 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 911/1808] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:36.685983 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb45748-0b01-4052-bd05-2b428f544969 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:36.696627 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb45748-0b01-4052-bd05-2b428f544969 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:36.703123 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b276607-4bc3-4f0f-9231-d06987bc3ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:36.704244 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b276607-4bc3-4f0f-9231-d06987bc3ddb admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:36.704605 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b276607-4bc3-4f0f-9231-d06987bc3ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:36.704953 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b276607-4bc3-4f0f-9231-d06987bc3ddb admin admin] PATH_INFO: `/v3/projects/08a8fd090dcf44028592fb533e635b09/users/ca0b82e3a8bc4143bf7053e1a66481b6/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:36.722096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b276607-4bc3-4f0f-9231-d06987bc3ddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=08a8fd090dcf44028592fb533e635b09, user_id=ca0b82e3a8bc4143bf7053e1a66481b6, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:36.731838 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b276607-4bc3-4f0f-9231-d06987bc3ddb admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:36.886956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 898/1809] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:36 2022] PUT /identity/v3/projects/08a8fd090dcf44028592fb533e635b09/users/ca0b82e3a8bc4143bf7053e1a66481b6/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:36.898844 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80efd7-9b73-4897-a048-299a9ab1c8d5 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:36.899382 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80efd7-9b73-4897-a048-299a9ab1c8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:36.899470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80efd7-9b73-4897-a048-299a9ab1c8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:36.921233 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:36.932062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-8b80efd7-9b73-4897-a048-299a9ab1c8d5 None None] MFA Rules not processed for user `ca0b82e3a8bc4143bf7053e1a66481b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:36.933489 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-8b80efd7-9b73-4897-a048-299a9ab1c8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:37.019863 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 912/1810] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:37.024811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b276607-4bc3-4f0f-9231-d06987bc3ddb admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:37.088727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b276607-4bc3-4f0f-9231-d06987bc3ddb admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:37.091040 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80efd7-9b73-4897-a048-299a9ab1c8d5 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:37.093384 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-8b80efd7-9b73-4897-a048-299a9ab1c8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:37.093718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98286d2a-0400-4bbb-8ab2-4ccd8dfe5e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:37.094393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98286d2a-0400-4bbb-8ab2-4ccd8dfe5e03 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:37.094678 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98286d2a-0400-4bbb-8ab2-4ccd8dfe5e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:37.094982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98286d2a-0400-4bbb-8ab2-4ccd8dfe5e03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:37.095695 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98286d2a-0400-4bbb-8ab2-4ccd8dfe5e03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:37.098894 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98286d2a-0400-4bbb-8ab2-4ccd8dfe5e03 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:37.112102 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80efd7-9b73-4897-a048-299a9ab1c8d5 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:37.117646 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc6041-d4e6-4296-a0f9-bcd4023b8491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:37.124976 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc6041-d4e6-4296-a0f9-bcd4023b8491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:37.124976 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc6041-d4e6-4296-a0f9-bcd4023b8491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:37.124976 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc6041-d4e6-4296-a0f9-bcd4023b8491 admin admin] PATH_INFO: `/v3/projects/cee70b6e066040cc852b69193661f0d6` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:37.124976 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc6041-d4e6-4296-a0f9-bcd4023b8491 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cee70b6e066040cc852b69193661f0d6)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:37.124976 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc6041-d4e6-4296-a0f9-bcd4023b8491 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:37.278155 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 899/1811] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:24:37 2022] POST /identity/v3/projects => generated 385 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:24:37.279430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98286d2a-0400-4bbb-8ab2-4ccd8dfe5e03 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:37.371478 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-cffc6041-d4e6-4296-a0f9-bcd4023b8491 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cee70b6e066040cc852b69193661f0d6', 'request_id': 'req-cffc6041-d4e6-4296-a0f9-bcd4023b8491'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:37.385001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-cffc6041-d4e6-4296-a0f9-bcd4023b8491 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cee70b6e066040cc852b69193661f0d6', 'request_id': 'req-cffc6041-d4e6-4296-a0f9-bcd4023b8491'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:37.403697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-cffc6041-d4e6-4296-a0f9-bcd4023b8491 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cee70b6e066040cc852b69193661f0d6', 'request_id': 'req-cffc6041-d4e6-4296-a0f9-bcd4023b8491'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:37.413778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98286d2a-0400-4bbb-8ab2-4ccd8dfe5e03 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:37.478896 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e11ed-6869-44df-858c-914299e42df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:37.479602 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e11ed-6869-44df-858c-914299e42df9 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:37.479887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e11ed-6869-44df-858c-914299e42df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:37.480164 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e11ed-6869-44df-858c-914299e42df9 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:37.480854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e11ed-6869-44df-858c-914299e42df9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:37.490535 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e11ed-6869-44df-858c-914299e42df9 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:37.601506 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-cffc6041-d4e6-4296-a0f9-bcd4023b8491 admin admin] The token cache is being invalidate because project cee70b6e066040cc852b69193661f0d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:24:37.601990 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-cffc6041-d4e6-4296-a0f9-bcd4023b8491 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:37.610866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-cffc6041-d4e6-4296-a0f9-bcd4023b8491 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:37.610866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 913/1812] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:24:37 2022] DELETE /identity/v3/projects/cee70b6e066040cc852b69193661f0d6 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:37.610866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797787b-9f1e-4982-97b4-723c8355a6c2 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:37.610866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797787b-9f1e-4982-97b4-723c8355a6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:37.610866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797787b-9f1e-4982-97b4-723c8355a6c2 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:37.610866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 914/1813] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jun 23 09:24:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:24:37.611368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccade53-4a0a-4a5b-a2f7-0dd5f28829e5 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:37.611368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccade53-4a0a-4a5b-a2f7-0dd5f28829e5 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:37.611368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccade53-4a0a-4a5b-a2f7-0dd5f28829e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:37.646960 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-6ccade53-4a0a-4a5b-a2f7-0dd5f28829e5 None None] MFA Rules not processed for user `7e9ecac5434e4d40806f57d53256f7fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:37.648663 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-6ccade53-4a0a-4a5b-a2f7-0dd5f28829e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:37.727518 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 915/1814] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:37.728497 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccade53-4a0a-4a5b-a2f7-0dd5f28829e5 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:37.729794 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-6ccade53-4a0a-4a5b-a2f7-0dd5f28829e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:37.743967 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccade53-4a0a-4a5b-a2f7-0dd5f28829e5 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:37.749453 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6876a5b-9a44-42ed-96f2-f85e891e6c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:37.750456 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6876a5b-9a44-42ed-96f2-f85e891e6c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:37.762348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6876a5b-9a44-42ed-96f2-f85e891e6c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:37.762348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6876a5b-9a44-42ed-96f2-f85e891e6c6d admin admin] PATH_INFO: `/v3/users/3c93baa0ed244642998a63289e775d6d` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:37.762348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6876a5b-9a44-42ed-96f2-f85e891e6c6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c93baa0ed244642998a63289e775d6d)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:37.763147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 900/1815] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:24:37 2022] POST /identity/v3/users => generated 299 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:37.763253 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6876a5b-9a44-42ed-96f2-f85e891e6c6d admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:37.763341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6cd49-a0e6-4aba-830f-f3437c231001 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:37.763341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6cd49-a0e6-4aba-830f-f3437c231001 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:37.763341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6cd49-a0e6-4aba-830f-f3437c231001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:37.779964 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-8de6cd49-a0e6-4aba-830f-f3437c231001 None None] MFA Rules not processed for user `ff025cb706d243778468ae4570374690`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:37.781196 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-8de6cd49-a0e6-4aba-830f-f3437c231001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:37.850323 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 901/1816] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jun 23 09:24:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:37.851197 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6cd49-a0e6-4aba-830f-f3437c231001 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:37.852557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-8de6cd49-a0e6-4aba-830f-f3437c231001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:37.866937 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6cd49-a0e6-4aba-830f-f3437c231001 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:37.871666 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181603f7-36a9-4bad-b52e-2162b8523dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:37.872951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181603f7-36a9-4bad-b52e-2162b8523dec admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:37.873247 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181603f7-36a9-4bad-b52e-2162b8523dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:37.873528 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181603f7-36a9-4bad-b52e-2162b8523dec admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:37.874233 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181603f7-36a9-4bad-b52e-2162b8523dec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:37.876172 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181603f7-36a9-4bad-b52e-2162b8523dec admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:38.114361 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 902/1817] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:24:37 2022] POST /identity/v3/projects => generated 373 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:38.114361 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181603f7-36a9-4bad-b52e-2162b8523dec admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:38.114361 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-181603f7-36a9-4bad-b52e-2162b8523dec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:38.179490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181603f7-36a9-4bad-b52e-2162b8523dec admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:38.184752 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d91030-6937-46b3-b9fa-31d4b81bad43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:38.185691 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d91030-6937-46b3-b9fa-31d4b81bad43 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:38.185980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d91030-6937-46b3-b9fa-31d4b81bad43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:38.186259 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d91030-6937-46b3-b9fa-31d4b81bad43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:38.187078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d91030-6937-46b3-b9fa-31d4b81bad43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:38.188886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d91030-6937-46b3-b9fa-31d4b81bad43 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:38.200700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 903/1818] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:38.214893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d91030-6937-46b3-b9fa-31d4b81bad43 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:38.214893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-33d91030-6937-46b3-b9fa-31d4b81bad43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:38.215140 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e6876a5b-9a44-42ed-96f2-f85e891e6c6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c93baa0ed244642998a63289e775d6d'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:38.220952 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e6876a5b-9a44-42ed-96f2-f85e891e6c6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c93baa0ed244642998a63289e775d6d'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:38.221669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e6876a5b-9a44-42ed-96f2-f85e891e6c6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c93baa0ed244642998a63289e775d6d'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:38.222332 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e6876a5b-9a44-42ed-96f2-f85e891e6c6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c93baa0ed244642998a63289e775d6d'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:38.239467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e6876a5b-9a44-42ed-96f2-f85e891e6c6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c93baa0ed244642998a63289e775d6d'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:38.283268 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d91030-6937-46b3-b9fa-31d4b81bad43 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:38.288819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19322e28-f30b-4b9c-aec1-2db1aec20786 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:38.289504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19322e28-f30b-4b9c-aec1-2db1aec20786 service nova] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:38.289796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19322e28-f30b-4b9c-aec1-2db1aec20786 service nova] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:38.290077 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19322e28-f30b-4b9c-aec1-2db1aec20786 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:38.291909 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-19322e28-f30b-4b9c-aec1-2db1aec20786 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:38.304798 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19322e28-f30b-4b9c-aec1-2db1aec20786 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:38.309618 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19322e28-f30b-4b9c-aec1-2db1aec20786 service nova] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:38.319311 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 904/1819] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:38.320283 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19322e28-f30b-4b9c-aec1-2db1aec20786 service nova] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:38.322340 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-19322e28-f30b-4b9c-aec1-2db1aec20786 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:38.346852 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19322e28-f30b-4b9c-aec1-2db1aec20786 service nova] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:38.354538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c34bf1-727c-41cf-8a00-e4fed834d860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:38.355364 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c34bf1-727c-41cf-8a00-e4fed834d860 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:38.355537 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c34bf1-727c-41cf-8a00-e4fed834d860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:38.355812 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c34bf1-727c-41cf-8a00-e4fed834d860 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:38.357118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c34bf1-727c-41cf-8a00-e4fed834d860 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:38.359351 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c34bf1-727c-41cf-8a00-e4fed834d860 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:38.382890 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 916/1820] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:24:37 2022] DELETE /identity/v3/users/3c93baa0ed244642998a63289e775d6d => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:38.382890 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6876a5b-9a44-42ed-96f2-f85e891e6c6d admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:38.382890 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e6876a5b-9a44-42ed-96f2-f85e891e6c6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:38.450493 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6876a5b-9a44-42ed-96f2-f85e891e6c6d admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:38.456181 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc761505-0c03-4199-a962-d46e994cf957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:38.457083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc761505-0c03-4199-a962-d46e994cf957 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:38.457371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc761505-0c03-4199-a962-d46e994cf957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:38.457659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc761505-0c03-4199-a962-d46e994cf957 admin admin] PATH_INFO: `/v3/projects/4d71447c8bb94fbca35ee4c0d1c778cc/users/70fe0a1fc7e14874a0800e90d376e7a9/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:38.478396 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc761505-0c03-4199-a962-d46e994cf957 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d71447c8bb94fbca35ee4c0d1c778cc, user_id=70fe0a1fc7e14874a0800e90d376e7a9, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:38.485139 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc761505-0c03-4199-a962-d46e994cf957 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:38.581873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 905/1821] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:24:38 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:38.584566 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c34bf1-727c-41cf-8a00-e4fed834d860 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:38.597284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c34bf1-727c-41cf-8a00-e4fed834d860 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:38.602012 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686f00a-d878-4c20-a52a-f4a433186411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:38.602685 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686f00a-d878-4c20-a52a-f4a433186411 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:38.603006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686f00a-d878-4c20-a52a-f4a433186411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:38.603290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686f00a-d878-4c20-a52a-f4a433186411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:38.604072 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c686f00a-d878-4c20-a52a-f4a433186411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:38.605773 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c686f00a-d878-4c20-a52a-f4a433186411 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:38.638489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 906/1822] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:38.638489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686f00a-d878-4c20-a52a-f4a433186411 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:38.638489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686f00a-d878-4c20-a52a-f4a433186411 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:38.642257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a30c2f-a12e-467f-8438-bd8baee8b204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:38.643302 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a30c2f-a12e-467f-8438-bd8baee8b204 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:38.643708 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a30c2f-a12e-467f-8438-bd8baee8b204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:38.644050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a30c2f-a12e-467f-8438-bd8baee8b204 admin admin] PATH_INFO: `/v3/projects/86d7646998114b4aa33caed11865f8a4/users/ca68ffe4044d44b8bdb7c326971d4b12/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:38.660390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a30c2f-a12e-467f-8438-bd8baee8b204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86d7646998114b4aa33caed11865f8a4, user_id=ca68ffe4044d44b8bdb7c326971d4b12, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:38.663382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a30c2f-a12e-467f-8438-bd8baee8b204 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:38.951822 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 917/1823] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:38 2022] PUT /identity/v3/projects/4d71447c8bb94fbca35ee4c0d1c778cc/users/70fe0a1fc7e14874a0800e90d376e7a9/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:38.952771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc761505-0c03-4199-a962-d46e994cf957 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:38.954938 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-dc761505-0c03-4199-a962-d46e994cf957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:38.993991 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 907/1824] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:38 2022] PUT /identity/v3/projects/86d7646998114b4aa33caed11865f8a4/users/ca68ffe4044d44b8bdb7c326971d4b12/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:38.993991 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a30c2f-a12e-467f-8438-bd8baee8b204 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:39.026953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc761505-0c03-4199-a962-d46e994cf957 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:39.099425 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a30c2f-a12e-467f-8438-bd8baee8b204 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:39.106591 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f209c86-46e4-401e-ae12-c38a52e6b396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:39.110012 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f209c86-46e4-401e-ae12-c38a52e6b396 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:39.110012 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f209c86-46e4-401e-ae12-c38a52e6b396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:39.110012 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f209c86-46e4-401e-ae12-c38a52e6b396 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:39.110012 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f209c86-46e4-401e-ae12-c38a52e6b396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:39.112151 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f209c86-46e4-401e-ae12-c38a52e6b396 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:39.112464 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b01a8e-f16a-480f-8e07-b629471829cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:39.113402 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b01a8e-f16a-480f-8e07-b629471829cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:39.113867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b01a8e-f16a-480f-8e07-b629471829cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:39.114158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b01a8e-f16a-480f-8e07-b629471829cf admin admin] PATH_INFO: `/v3/users/2a31c92089cf47aca10c399dafe6aa55` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:39.119616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b01a8e-f16a-480f-8e07-b629471829cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a31c92089cf47aca10c399dafe6aa55)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:39.122593 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b01a8e-f16a-480f-8e07-b629471829cf admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:39.125797 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 908/1825] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:39.128556 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f209c86-46e4-401e-ae12-c38a52e6b396 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:39.140707 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f209c86-46e4-401e-ae12-c38a52e6b396 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:39.145916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d55f9-7e63-4394-a448-4b77a71d47a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:39.146668 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d55f9-7e63-4394-a448-4b77a71d47a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:39.146953 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d55f9-7e63-4394-a448-4b77a71d47a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:39.147653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d55f9-7e63-4394-a448-4b77a71d47a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:39.148412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d55f9-7e63-4394-a448-4b77a71d47a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:39.150093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d55f9-7e63-4394-a448-4b77a71d47a0 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:39.163862 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 909/1826] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:39.165019 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d55f9-7e63-4394-a448-4b77a71d47a0 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:39.190237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d55f9-7e63-4394-a448-4b77a71d47a0 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:39.197595 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b924d43-ee53-4527-997d-ed1f0292b231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:39.198863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b924d43-ee53-4527-997d-ed1f0292b231 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:39.199379 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b924d43-ee53-4527-997d-ed1f0292b231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:39.199895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b924d43-ee53-4527-997d-ed1f0292b231 admin admin] PATH_INFO: `/v3/projects/4d71447c8bb94fbca35ee4c0d1c778cc/users/70fe0a1fc7e14874a0800e90d376e7a9/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:39.202110 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b924d43-ee53-4527-997d-ed1f0292b231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d71447c8bb94fbca35ee4c0d1c778cc, user_id=70fe0a1fc7e14874a0800e90d376e7a9, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:39.205306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b924d43-ee53-4527-997d-ed1f0292b231 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:39.221660 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 910/1827] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:39 2022] PUT /identity/v3/projects/4d71447c8bb94fbca35ee4c0d1c778cc/users/70fe0a1fc7e14874a0800e90d376e7a9/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:39.230469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b924d43-ee53-4527-997d-ed1f0292b231 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:39.328090 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b924d43-ee53-4527-997d-ed1f0292b231 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:39.333280 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4c3bc-2fb4-4255-ab6f-d38e817dacdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:39.334304 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4c3bc-2fb4-4255-ab6f-d38e817dacdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:39.334697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4c3bc-2fb4-4255-ab6f-d38e817dacdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:39.335130 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4c3bc-2fb4-4255-ab6f-d38e817dacdd admin admin] PATH_INFO: `/v3/projects/86d7646998114b4aa33caed11865f8a4/users/ca68ffe4044d44b8bdb7c326971d4b12/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:39.338058 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4c3bc-2fb4-4255-ab6f-d38e817dacdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=86d7646998114b4aa33caed11865f8a4, user_id=ca68ffe4044d44b8bdb7c326971d4b12, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:39.341849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4c3bc-2fb4-4255-ab6f-d38e817dacdd admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:39.352943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 911/1828] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:39 2022] PUT /identity/v3/projects/86d7646998114b4aa33caed11865f8a4/users/ca68ffe4044d44b8bdb7c326971d4b12/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:39.358253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12c473-fd34-4f3c-96db-51de9070f379 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:39.358555 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12c473-fd34-4f3c-96db-51de9070f379 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:39.358844 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12c473-fd34-4f3c-96db-51de9070f379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:39.397322 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e1b01a8e-f16a-480f-8e07-b629471829cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a31c92089cf47aca10c399dafe6aa55'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:39.416604 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e1b01a8e-f16a-480f-8e07-b629471829cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a31c92089cf47aca10c399dafe6aa55'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:39.417986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e1b01a8e-f16a-480f-8e07-b629471829cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a31c92089cf47aca10c399dafe6aa55'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:39.420718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e1b01a8e-f16a-480f-8e07-b629471829cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a31c92089cf47aca10c399dafe6aa55'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:39.428420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-cf12c473-fd34-4f3c-96db-51de9070f379 None None] MFA Rules not processed for user `70fe0a1fc7e14874a0800e90d376e7a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:39.429660 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-cf12c473-fd34-4f3c-96db-51de9070f379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:39.441739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e1b01a8e-f16a-480f-8e07-b629471829cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a31c92089cf47aca10c399dafe6aa55'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:39.520855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 912/1829] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:39.520855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12c473-fd34-4f3c-96db-51de9070f379 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:39.520855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-cf12c473-fd34-4f3c-96db-51de9070f379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:39.610604 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12c473-fd34-4f3c-96db-51de9070f379 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:39.616973 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 918/1830] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:24:38 2022] DELETE /identity/v3/users/2a31c92089cf47aca10c399dafe6aa55 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:39.618340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b01a8e-f16a-480f-8e07-b629471829cf admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:39.620041 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e1b01a8e-f16a-480f-8e07-b629471829cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:39.623303 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932e519-98bf-4343-a954-dc98c4b6294d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:39.624349 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932e519-98bf-4343-a954-dc98c4b6294d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:39.624635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932e519-98bf-4343-a954-dc98c4b6294d admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:39.626021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932e519-98bf-4343-a954-dc98c4b6294d admin admin] PATH_INFO: `/v3/projects/e18c722103c144378f8498bbd8c11f82` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:39.632741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932e519-98bf-4343-a954-dc98c4b6294d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e18c722103c144378f8498bbd8c11f82)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:39.632741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932e519-98bf-4343-a954-dc98c4b6294d admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:39.721058 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b01a8e-f16a-480f-8e07-b629471829cf admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:39.724697 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66c608-fa32-45e2-835c-536f8be937bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:39.728192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66c608-fa32-45e2-835c-536f8be937bd service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:39.728192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66c608-fa32-45e2-835c-536f8be937bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:39.728192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66c608-fa32-45e2-835c-536f8be937bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:39.808718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66c608-fa32-45e2-835c-536f8be937bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:39.811102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-5932e519-98bf-4343-a954-dc98c4b6294d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e18c722103c144378f8498bbd8c11f82', 'request_id': 'req-5932e519-98bf-4343-a954-dc98c4b6294d'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:39.813004 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66c608-fa32-45e2-835c-536f8be937bd service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:39.821162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-5932e519-98bf-4343-a954-dc98c4b6294d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e18c722103c144378f8498bbd8c11f82', 'request_id': 'req-5932e519-98bf-4343-a954-dc98c4b6294d'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:39.831245 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 919/1831] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:39.836354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f017a8-219b-41eb-9054-139aba11c57a None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:39.836963 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f017a8-219b-41eb-9054-139aba11c57a None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:39.837424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f017a8-219b-41eb-9054-139aba11c57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:39.870404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-05f017a8-219b-41eb-9054-139aba11c57a None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:39.870404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-05f017a8-219b-41eb-9054-139aba11c57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:39.926268 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-5932e519-98bf-4343-a954-dc98c4b6294d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e18c722103c144378f8498bbd8c11f82', 'request_id': 'req-5932e519-98bf-4343-a954-dc98c4b6294d'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:39.939303 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-5932e519-98bf-4343-a954-dc98c4b6294d admin admin] The token cache is being invalidate because project e18c722103c144378f8498bbd8c11f82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:24:39.940356 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-5932e519-98bf-4343-a954-dc98c4b6294d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:39.941352 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-5932e519-98bf-4343-a954-dc98c4b6294d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:39.943564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 913/1832] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:24:39 2022] DELETE /identity/v3/projects/e18c722103c144378f8498bbd8c11f82 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:39.944757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932e519-98bf-4343-a954-dc98c4b6294d admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:39.946257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5932e519-98bf-4343-a954-dc98c4b6294d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:40.009747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 920/1833] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:40.009747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c5b1e-de1e-4266-ba2f-4ff26ec675a4 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:40.009747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c5b1e-de1e-4266-ba2f-4ff26ec675a4 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:40.009747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c5b1e-de1e-4266-ba2f-4ff26ec675a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:40.030381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-b09c5b1e-de1e-4266-ba2f-4ff26ec675a4 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:40.031720 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b09c5b1e-de1e-4266-ba2f-4ff26ec675a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:40.038404 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932e519-98bf-4343-a954-dc98c4b6294d admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:40.046457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 921/1834] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:40.046457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c5b1e-de1e-4266-ba2f-4ff26ec675a4 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:40.046706 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4af56e-cb4b-4819-8329-9bd5bdd397e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:40.046706 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4af56e-cb4b-4819-8329-9bd5bdd397e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:40.046706 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4af56e-cb4b-4819-8329-9bd5bdd397e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:40.046706 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4af56e-cb4b-4819-8329-9bd5bdd397e6 admin admin] PATH_INFO: `/v3/users/ca0b82e3a8bc4143bf7053e1a66481b6` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:40.047107 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4af56e-cb4b-4819-8329-9bd5bdd397e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca0b82e3a8bc4143bf7053e1a66481b6)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:40.048463 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4af56e-cb4b-4819-8329-9bd5bdd397e6 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:40.049720 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b09c5b1e-de1e-4266-ba2f-4ff26ec675a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:40.120913 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c5b1e-de1e-4266-ba2f-4ff26ec675a4 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:40.125989 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540e6e4-e3b8-4eb3-a4d2-da6ebf01d7b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:40.126709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540e6e4-e3b8-4eb3-a4d2-da6ebf01d7b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:40.127036 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540e6e4-e3b8-4eb3-a4d2-da6ebf01d7b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:40.127325 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540e6e4-e3b8-4eb3-a4d2-da6ebf01d7b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:40.139238 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b540e6e4-e3b8-4eb3-a4d2-da6ebf01d7b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:40.140772 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b540e6e4-e3b8-4eb3-a4d2-da6ebf01d7b2 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:40.153668 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 922/1835] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:40.155264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09585754-30e1-407a-a261-1180427fe9ff None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:40.156030 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09585754-30e1-407a-a261-1180427fe9ff None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:40.156030 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09585754-30e1-407a-a261-1180427fe9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:40.177886 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-09585754-30e1-407a-a261-1180427fe9ff None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:40.179235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-09585754-30e1-407a-a261-1180427fe9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:40.197128 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 923/1836] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:40.202087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09585754-30e1-407a-a261-1180427fe9ff None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:40.226764 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09585754-30e1-407a-a261-1180427fe9ff None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:40.232386 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a36a0b-8dc4-4b19-9db3-49c51aacce5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:40.236318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a36a0b-8dc4-4b19-9db3-49c51aacce5a admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:40.236318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a36a0b-8dc4-4b19-9db3-49c51aacce5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:40.236318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a36a0b-8dc4-4b19-9db3-49c51aacce5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:40.236318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a36a0b-8dc4-4b19-9db3-49c51aacce5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:24:40.236318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a36a0b-8dc4-4b19-9db3-49c51aacce5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:40.236686 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a36a0b-8dc4-4b19-9db3-49c51aacce5a admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:40.253200 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 924/1837] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:24:40.258244 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a36a0b-8dc4-4b19-9db3-49c51aacce5a admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:40.273030 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a36a0b-8dc4-4b19-9db3-49c51aacce5a admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:40.281062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d4e79-554a-42ae-996b-93c85a6c31a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:40.281818 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d4e79-554a-42ae-996b-93c85a6c31a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:40.282337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d4e79-554a-42ae-996b-93c85a6c31a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:40.282847 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d4e79-554a-42ae-996b-93c85a6c31a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:40.283675 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d4e79-554a-42ae-996b-93c85a6c31a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:40.285671 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d4e79-554a-42ae-996b-93c85a6c31a6 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:40.307443 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-ac4af56e-cb4b-4819-8329-9bd5bdd397e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca0b82e3a8bc4143bf7053e1a66481b6'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:40.315672 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-ac4af56e-cb4b-4819-8329-9bd5bdd397e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca0b82e3a8bc4143bf7053e1a66481b6'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:40.316420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-ac4af56e-cb4b-4819-8329-9bd5bdd397e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca0b82e3a8bc4143bf7053e1a66481b6'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:40.317036 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-ac4af56e-cb4b-4819-8329-9bd5bdd397e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca0b82e3a8bc4143bf7053e1a66481b6'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:40.336216 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-ac4af56e-cb4b-4819-8329-9bd5bdd397e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca0b82e3a8bc4143bf7053e1a66481b6'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:40.414680 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 925/1838] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:24:40 2022] POST /identity/v3/projects => generated 377 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:40.418545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d4e79-554a-42ae-996b-93c85a6c31a6 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:40.420898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-ac6d4e79-554a-42ae-996b-93c85a6c31a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:40.445993 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 914/1839] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:24:39 2022] DELETE /identity/v3/users/ca0b82e3a8bc4143bf7053e1a66481b6 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:40.451172 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff93c6-6602-4f38-8dea-74c628f9d4c7 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:40.452029 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff93c6-6602-4f38-8dea-74c628f9d4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:40.452319 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff93c6-6602-4f38-8dea-74c628f9d4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:40.484265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-38ff93c6-6602-4f38-8dea-74c628f9d4c7 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:40.487295 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-38ff93c6-6602-4f38-8dea-74c628f9d4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:40.520798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d4e79-554a-42ae-996b-93c85a6c31a6 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:40.550746 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:40.591764 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 915/1840] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:24:40.603291 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146162df-d081-4762-9369-5ea38ca4a650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:40.606800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146162df-d081-4762-9369-5ea38ca4a650 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:40.606800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146162df-d081-4762-9369-5ea38ca4a650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:40.606800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146162df-d081-4762-9369-5ea38ca4a650 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:40.606800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146162df-d081-4762-9369-5ea38ca4a650 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:40.607774 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff93c6-6602-4f38-8dea-74c628f9d4c7 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:40.608281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146162df-d081-4762-9369-5ea38ca4a650 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:40.609422 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-38ff93c6-6602-4f38-8dea-74c628f9d4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:40.681266 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff93c6-6602-4f38-8dea-74c628f9d4c7 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:40.686160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e235b-7b14-49fb-aae9-f792595894b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:40.686595 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e235b-7b14-49fb-aae9-f792595894b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:40.686932 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e235b-7b14-49fb-aae9-f792595894b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:40.687219 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e235b-7b14-49fb-aae9-f792595894b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:40.698095 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e235b-7b14-49fb-aae9-f792595894b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:40.699586 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e235b-7b14-49fb-aae9-f792595894b1 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:40.710002 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 916/1841] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:40.734328 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 926/1842] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:24:40 2022] POST /identity/v3/users => generated 289 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:40.735586 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e235b-7b14-49fb-aae9-f792595894b1 service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:40.747568 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e235b-7b14-49fb-aae9-f792595894b1 service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:40.753786 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268789b1-06e0-40dd-bf65-edb682caa11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:40.754469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268789b1-06e0-40dd-bf65-edb682caa11e admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:40.754758 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268789b1-06e0-40dd-bf65-edb682caa11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:40.755077 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268789b1-06e0-40dd-bf65-edb682caa11e admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:40.756838 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268789b1-06e0-40dd-bf65-edb682caa11e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:40.758197 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268789b1-06e0-40dd-bf65-edb682caa11e admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:40.781769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 917/1843] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:40.783058 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146162df-d081-4762-9369-5ea38ca4a650 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:40.784935 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-146162df-d081-4762-9369-5ea38ca4a650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:40.787877 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268789b1-06e0-40dd-bf65-edb682caa11e admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:40.798940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268789b1-06e0-40dd-bf65-edb682caa11e admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:40.804544 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa226411-8fd4-4914-afc9-8433d94ecbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:40.805229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa226411-8fd4-4914-afc9-8433d94ecbf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:40.806099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa226411-8fd4-4914-afc9-8433d94ecbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:40.806099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa226411-8fd4-4914-afc9-8433d94ecbf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:40.806863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa226411-8fd4-4914-afc9-8433d94ecbf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:40.808540 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa226411-8fd4-4914-afc9-8433d94ecbf2 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:40.813739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146162df-d081-4762-9369-5ea38ca4a650 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:40.819521 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 918/1844] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:40.824208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa226411-8fd4-4914-afc9-8433d94ecbf2 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:40.826815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83463479-46ba-4ac4-aed5-21fe70f94ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:40.828179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83463479-46ba-4ac4-aed5-21fe70f94ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:40.828542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83463479-46ba-4ac4-aed5-21fe70f94ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:40.829269 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83463479-46ba-4ac4-aed5-21fe70f94ea1 admin admin] PATH_INFO: `/v3/projects/8e9ba71662eb4901b4aeae33eed1ecd7` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:40.830619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83463479-46ba-4ac4-aed5-21fe70f94ea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e9ba71662eb4901b4aeae33eed1ecd7)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:40.832893 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83463479-46ba-4ac4-aed5-21fe70f94ea1 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:40.848690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa226411-8fd4-4914-afc9-8433d94ecbf2 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:40.854166 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f80e2-07c9-4a96-87ac-c96cd3c3670a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:40.857901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f80e2-07c9-4a96-87ac-c96cd3c3670a admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:40.857901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f80e2-07c9-4a96-87ac-c96cd3c3670a admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:40.857901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f80e2-07c9-4a96-87ac-c96cd3c3670a admin admin] PATH_INFO: `/v3/projects/cf8494b864f24d89ac729bd9fd8517c7/users/2d0abc022f6f4a1d9a44375791e38663/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:40.872995 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f80e2-07c9-4a96-87ac-c96cd3c3670a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf8494b864f24d89ac729bd9fd8517c7, user_id=2d0abc022f6f4a1d9a44375791e38663, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:40.875862 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f80e2-07c9-4a96-87ac-c96cd3c3670a admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:41.040221 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-83463479-46ba-4ac4-aed5-21fe70f94ea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e9ba71662eb4901b4aeae33eed1ecd7', 'request_id': 'req-83463479-46ba-4ac4-aed5-21fe70f94ea1'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:41.058343 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 919/1845] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:40 2022] PUT /identity/v3/projects/cf8494b864f24d89ac729bd9fd8517c7/users/2d0abc022f6f4a1d9a44375791e38663/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:41.058709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-83463479-46ba-4ac4-aed5-21fe70f94ea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e9ba71662eb4901b4aeae33eed1ecd7', 'request_id': 'req-83463479-46ba-4ac4-aed5-21fe70f94ea1'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:41.058812 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc11781-eec7-4b04-a91e-9d56d083431a None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:41.058812 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc11781-eec7-4b04-a91e-9d56d083431a None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:41.058812 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc11781-eec7-4b04-a91e-9d56d083431a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:41.060839 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-83463479-46ba-4ac4-aed5-21fe70f94ea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e9ba71662eb4901b4aeae33eed1ecd7', 'request_id': 'req-83463479-46ba-4ac4-aed5-21fe70f94ea1'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:41.129033 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-ecc11781-eec7-4b04-a91e-9d56d083431a None None] MFA Rules not processed for user `2d0abc022f6f4a1d9a44375791e38663`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:41.130029 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ecc11781-eec7-4b04-a91e-9d56d083431a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:41.259173 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-83463479-46ba-4ac4-aed5-21fe70f94ea1 admin admin] The token cache is being invalidate because project 8e9ba71662eb4901b4aeae33eed1ecd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:24:41.259684 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-83463479-46ba-4ac4-aed5-21fe70f94ea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:41.267178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-83463479-46ba-4ac4-aed5-21fe70f94ea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:41.268200 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 920/1846] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:41.269626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23723970-6b0e-476f-a803-9f170dee85aa None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:41.269917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23723970-6b0e-476f-a803-9f170dee85aa None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:41.270225 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23723970-6b0e-476f-a803-9f170dee85aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:41.276378 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 927/1847] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:24:40 2022] DELETE /identity/v3/projects/8e9ba71662eb4901b4aeae33eed1ecd7 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:24:41.279905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83463479-46ba-4ac4-aed5-21fe70f94ea1 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:41.281603 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-83463479-46ba-4ac4-aed5-21fe70f94ea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:41.304059 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-23723970-6b0e-476f-a803-9f170dee85aa None None] MFA Rules not processed for user `ca68ffe4044d44b8bdb7c326971d4b12`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:41.305469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-23723970-6b0e-476f-a803-9f170dee85aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:41.373481 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83463479-46ba-4ac4-aed5-21fe70f94ea1 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:41.378162 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d054bf8-fb98-485c-9f0b-26d693c21f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:41.379080 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d054bf8-fb98-485c-9f0b-26d693c21f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:41.379375 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d054bf8-fb98-485c-9f0b-26d693c21f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:41.379656 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d054bf8-fb98-485c-9f0b-26d693c21f9a admin admin] PATH_INFO: `/v3/users/aa441aefdec64a74ab2f0aa1554433a9` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:41.380859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d054bf8-fb98-485c-9f0b-26d693c21f9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa441aefdec64a74ab2f0aa1554433a9)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:41.383022 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d054bf8-fb98-485c-9f0b-26d693c21f9a admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:41.399939 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 921/1848] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:41 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:41.400477 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23723970-6b0e-476f-a803-9f170dee85aa None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:41.402110 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-23723970-6b0e-476f-a803-9f170dee85aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:41.447379 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:41.468877 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23723970-6b0e-476f-a803-9f170dee85aa None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:41.472616 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b3b93-ae09-4677-9b60-4a17ec4f4196 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36a6a3190dad4948a505bbe4c7095699', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:41.473378 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b3b93-ae09-4677-9b60-4a17ec4f4196 service glance] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:41.473577 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b3b93-ae09-4677-9b60-4a17ec4f4196 service glance] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:41.474010 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b3b93-ae09-4677-9b60-4a17ec4f4196 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:41.476041 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-609b3b93-ae09-4677-9b60-4a17ec4f4196 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:41.488294 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b3b93-ae09-4677-9b60-4a17ec4f4196 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:41.489663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b3b93-ae09-4677-9b60-4a17ec4f4196 service glance] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:41.501284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 922/1849] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:41.502158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b3b93-ae09-4677-9b60-4a17ec4f4196 service glance] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:41.503538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-609b3b93-ae09-4677-9b60-4a17ec4f4196 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:41.519453 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b3b93-ae09-4677-9b60-4a17ec4f4196 service glance] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:41.524924 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c776c-b598-426e-8f95-3cb0556380ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:41.525537 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c776c-b598-426e-8f95-3cb0556380ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:41.525824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c776c-b598-426e-8f95-3cb0556380ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:41.526101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c776c-b598-426e-8f95-3cb0556380ee admin admin] PATH_INFO: `/v3/projects/08a8fd090dcf44028592fb533e635b09` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:41.527511 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c776c-b598-426e-8f95-3cb0556380ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=08a8fd090dcf44028592fb533e635b09)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:41.530873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c776c-b598-426e-8f95-3cb0556380ee admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:41.742300 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-be2c776c-b598-426e-8f95-3cb0556380ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08a8fd090dcf44028592fb533e635b09', 'request_id': 'req-be2c776c-b598-426e-8f95-3cb0556380ee'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:41.745698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-2d054bf8-fb98-485c-9f0b-26d693c21f9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa441aefdec64a74ab2f0aa1554433a9'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:41.750671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-be2c776c-b598-426e-8f95-3cb0556380ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08a8fd090dcf44028592fb533e635b09', 'request_id': 'req-be2c776c-b598-426e-8f95-3cb0556380ee'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:41.771824 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-2d054bf8-fb98-485c-9f0b-26d693c21f9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa441aefdec64a74ab2f0aa1554433a9'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:41.772870 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-2d054bf8-fb98-485c-9f0b-26d693c21f9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa441aefdec64a74ab2f0aa1554433a9'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:41.773528 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-2d054bf8-fb98-485c-9f0b-26d693c21f9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa441aefdec64a74ab2f0aa1554433a9'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:41.795348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-2d054bf8-fb98-485c-9f0b-26d693c21f9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa441aefdec64a74ab2f0aa1554433a9'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:41.847049 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-be2c776c-b598-426e-8f95-3cb0556380ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08a8fd090dcf44028592fb533e635b09', 'request_id': 'req-be2c776c-b598-426e-8f95-3cb0556380ee'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:41.859993 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-be2c776c-b598-426e-8f95-3cb0556380ee admin admin] The token cache is being invalidate because project 08a8fd090dcf44028592fb533e635b09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:24:41.860285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-be2c776c-b598-426e-8f95-3cb0556380ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:41.860946 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-be2c776c-b598-426e-8f95-3cb0556380ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:41.862854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 923/1850] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:24:41 2022] DELETE /identity/v3/projects/08a8fd090dcf44028592fb533e635b09 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:41.863778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c776c-b598-426e-8f95-3cb0556380ee admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:41.865350 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-be2c776c-b598-426e-8f95-3cb0556380ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:41.923542 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c776c-b598-426e-8f95-3cb0556380ee admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:41.927357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc95ffc-0d3b-4a20-ac01-8037faf89f23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:41.928011 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc95ffc-0d3b-4a20-ac01-8037faf89f23 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:41.928306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc95ffc-0d3b-4a20-ac01-8037faf89f23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:41.928613 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc95ffc-0d3b-4a20-ac01-8037faf89f23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:41.930334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-7fc95ffc-0d3b-4a20-ac01-8037faf89f23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:41.933751 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 928/1851] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:24:41 2022] DELETE /identity/v3/users/aa441aefdec64a74ab2f0aa1554433a9 => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:41.934739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d054bf8-fb98-485c-9f0b-26d693c21f9a admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:41.937721 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-2d054bf8-fb98-485c-9f0b-26d693c21f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:42.030232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d054bf8-fb98-485c-9f0b-26d693c21f9a admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:42.036547 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25da42-5b84-49db-833e-0aedd7814a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:42.038189 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25da42-5b84-49db-833e-0aedd7814a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:42.038630 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25da42-5b84-49db-833e-0aedd7814a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:42.039226 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25da42-5b84-49db-833e-0aedd7814a61 admin admin] PATH_INFO: `/v3/users/70fe0a1fc7e14874a0800e90d376e7a9` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:42.041655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25da42-5b84-49db-833e-0aedd7814a61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70fe0a1fc7e14874a0800e90d376e7a9)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:42.045331 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25da42-5b84-49db-833e-0aedd7814a61 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:42.136319 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc95ffc-0d3b-4a20-ac01-8037faf89f23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:42.137791 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc95ffc-0d3b-4a20-ac01-8037faf89f23 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:42.146774 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 924/1852] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:42.422847 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-de25da42-5b84-49db-833e-0aedd7814a61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70fe0a1fc7e14874a0800e90d376e7a9'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:42.440425 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-de25da42-5b84-49db-833e-0aedd7814a61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70fe0a1fc7e14874a0800e90d376e7a9'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:42.441534 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-de25da42-5b84-49db-833e-0aedd7814a61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70fe0a1fc7e14874a0800e90d376e7a9'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:42.442197 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-de25da42-5b84-49db-833e-0aedd7814a61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70fe0a1fc7e14874a0800e90d376e7a9'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:42.464216 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-de25da42-5b84-49db-833e-0aedd7814a61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70fe0a1fc7e14874a0800e90d376e7a9'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:42.555957 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 929/1853] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:24:41 2022] DELETE /identity/v3/users/70fe0a1fc7e14874a0800e90d376e7a9 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:42.646181 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc95ffc-0d3b-4a20-ac01-8037faf89f23 service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:42.648290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-7fc95ffc-0d3b-4a20-ac01-8037faf89f23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:42.777315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc95ffc-0d3b-4a20-ac01-8037faf89f23 service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:42.884691 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1ee0c-0b98-4cb3-bdd0-e254e8e17772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:42.885815 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1ee0c-0b98-4cb3-bdd0-e254e8e17772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:42.886543 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1ee0c-0b98-4cb3-bdd0-e254e8e17772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:42.887035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1ee0c-0b98-4cb3-bdd0-e254e8e17772 admin admin] PATH_INFO: `/v3/projects/e69f07e6433b48a28cb628a84639a85c` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:42.889199 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1ee0c-0b98-4cb3-bdd0-e254e8e17772 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e69f07e6433b48a28cb628a84639a85c)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:42.891392 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1ee0c-0b98-4cb3-bdd0-e254e8e17772 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:43.122306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4ee1ee0c-0b98-4cb3-bdd0-e254e8e17772 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e69f07e6433b48a28cb628a84639a85c', 'request_id': 'req-4ee1ee0c-0b98-4cb3-bdd0-e254e8e17772'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:43.140008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4ee1ee0c-0b98-4cb3-bdd0-e254e8e17772 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e69f07e6433b48a28cb628a84639a85c', 'request_id': 'req-4ee1ee0c-0b98-4cb3-bdd0-e254e8e17772'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:43.377551 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4ee1ee0c-0b98-4cb3-bdd0-e254e8e17772 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e69f07e6433b48a28cb628a84639a85c', 'request_id': 'req-4ee1ee0c-0b98-4cb3-bdd0-e254e8e17772'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:43.391099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4ee1ee0c-0b98-4cb3-bdd0-e254e8e17772 admin admin] The token cache is being invalidate because project e69f07e6433b48a28cb628a84639a85c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:24:43.392038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4ee1ee0c-0b98-4cb3-bdd0-e254e8e17772 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:43.392732 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4ee1ee0c-0b98-4cb3-bdd0-e254e8e17772 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:43.394827 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 925/1854] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:24:42 2022] DELETE /identity/v3/projects/e69f07e6433b48a28cb628a84639a85c => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:43.400737 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec76941-910b-45ac-b85c-94444af75b90 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:43.400993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec76941-910b-45ac-b85c-94444af75b90 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:43.402161 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec76941-910b-45ac-b85c-94444af75b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:43.467424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-5ec76941-910b-45ac-b85c-94444af75b90 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:43.468942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-5ec76941-910b-45ac-b85c-94444af75b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:43.479518 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1ee0c-0b98-4cb3-bdd0-e254e8e17772 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:43.480096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-4ee1ee0c-0b98-4cb3-bdd0-e254e8e17772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:43.601740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 930/1855] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:43.604231 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0e80c-f84d-4bba-8c2c-0667aab8e840 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:43.608190 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0e80c-f84d-4bba-8c2c-0667aab8e840 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:43.608888 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0e80c-f84d-4bba-8c2c-0667aab8e840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:43.612021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1ee0c-0b98-4cb3-bdd0-e254e8e17772 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:43.619211 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9617ba-2d8d-4209-a69a-743da0262770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:43.620114 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9617ba-2d8d-4209-a69a-743da0262770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:43.620402 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9617ba-2d8d-4209-a69a-743da0262770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:43.620741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9617ba-2d8d-4209-a69a-743da0262770 admin admin] PATH_INFO: `/v3/projects/4d71447c8bb94fbca35ee4c0d1c778cc` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:43.622589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9617ba-2d8d-4209-a69a-743da0262770 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d71447c8bb94fbca35ee4c0d1c778cc)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:43.625695 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9617ba-2d8d-4209-a69a-743da0262770 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:43.688143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-07d0e80c-f84d-4bba-8c2c-0667aab8e840 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:43.689723 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-07d0e80c-f84d-4bba-8c2c-0667aab8e840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:43.704786 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 931/1856] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:43.709428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0e80c-f84d-4bba-8c2c-0667aab8e840 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:43.729083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0e80c-f84d-4bba-8c2c-0667aab8e840 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:43.738555 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9678232-ba89-4152-968f-eaaef0737465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:43.740092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9678232-ba89-4152-968f-eaaef0737465 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:43.740631 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9678232-ba89-4152-968f-eaaef0737465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:43.741669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9678232-ba89-4152-968f-eaaef0737465 admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:43.742972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9678232-ba89-4152-968f-eaaef0737465 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:24:43.744254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9678232-ba89-4152-968f-eaaef0737465 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:43.747283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9678232-ba89-4152-968f-eaaef0737465 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:43.770452 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-3f9617ba-2d8d-4209-a69a-743da0262770 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d71447c8bb94fbca35ee4c0d1c778cc', 'request_id': 'req-3f9617ba-2d8d-4209-a69a-743da0262770'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:43.783258 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-3f9617ba-2d8d-4209-a69a-743da0262770 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d71447c8bb94fbca35ee4c0d1c778cc', 'request_id': 'req-3f9617ba-2d8d-4209-a69a-743da0262770'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:43.796486 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 932/1857] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:24:43.801362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9678232-ba89-4152-968f-eaaef0737465 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:43.826705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9678232-ba89-4152-968f-eaaef0737465 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:43.918800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47a1f0-0997-4ef8-8f6f-45434c918f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:43.919551 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47a1f0-0997-4ef8-8f6f-45434c918f62 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:43.919856 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47a1f0-0997-4ef8-8f6f-45434c918f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:43.920139 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47a1f0-0997-4ef8-8f6f-45434c918f62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:43.920853 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac47a1f0-0997-4ef8-8f6f-45434c918f62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:43.920976 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-3f9617ba-2d8d-4209-a69a-743da0262770 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d71447c8bb94fbca35ee4c0d1c778cc', 'request_id': 'req-3f9617ba-2d8d-4209-a69a-743da0262770'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:43.922792 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac47a1f0-0997-4ef8-8f6f-45434c918f62 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:43.941321 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-3f9617ba-2d8d-4209-a69a-743da0262770 admin admin] The token cache is being invalidate because project 4d71447c8bb94fbca35ee4c0d1c778cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:24:43.941639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-3f9617ba-2d8d-4209-a69a-743da0262770 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:43.942309 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-3f9617ba-2d8d-4209-a69a-743da0262770 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:43.944421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 926/1858] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:24:43 2022] DELETE /identity/v3/projects/4d71447c8bb94fbca35ee4c0d1c778cc => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:24:43.949194 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15869289-de7b-47c7-b639-8a1d1ecb5573 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:43.950376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15869289-de7b-47c7-b639-8a1d1ecb5573 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:43.950878 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15869289-de7b-47c7-b639-8a1d1ecb5573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:43.972728 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-15869289-de7b-47c7-b639-8a1d1ecb5573 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:43.974102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-15869289-de7b-47c7-b639-8a1d1ecb5573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:43.984648 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 927/1859] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:24:43.992711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac0562-4ec9-4919-9e70-ad10b059dfa8 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:43.993247 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac0562-4ec9-4919-9e70-ad10b059dfa8 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:43.993687 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac0562-4ec9-4919-9e70-ad10b059dfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:44.019076 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-34ac0562-4ec9-4919-9e70-ad10b059dfa8 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:44.021009 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-34ac0562-4ec9-4919-9e70-ad10b059dfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:44.030725 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 928/1860] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:44.034498 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac0562-4ec9-4919-9e70-ad10b059dfa8 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:44.045751 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac0562-4ec9-4919-9e70-ad10b059dfa8 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:44.064841 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a671f-1a48-4623-a357-909195f6119a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:44.064841 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a671f-1a48-4623-a357-909195f6119a admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:44.064841 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a671f-1a48-4623-a357-909195f6119a admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:44.064841 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a671f-1a48-4623-a357-909195f6119a admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:44.065271 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a671f-1a48-4623-a357-909195f6119a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:24:44.065271 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a671f-1a48-4623-a357-909195f6119a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:44.065271 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a671f-1a48-4623-a357-909195f6119a admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:44.068018 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 933/1861] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:24:43 2022] POST /identity/v3/projects => generated 387 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:44.076629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 929/1862] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:24:44.079922 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47a1f0-0997-4ef8-8f6f-45434c918f62 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:44.081192 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a671f-1a48-4623-a357-909195f6119a admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:44.081731 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-ac47a1f0-0997-4ef8-8f6f-45434c918f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:44.107061 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a671f-1a48-4623-a357-909195f6119a admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:44.157392 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47a1f0-0997-4ef8-8f6f-45434c918f62 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:44.219311 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef493891-2adf-4d1a-b467-074f59201c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:44.219311 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef493891-2adf-4d1a-b467-074f59201c8f admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:44.219311 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef493891-2adf-4d1a-b467-074f59201c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:44.219311 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef493891-2adf-4d1a-b467-074f59201c8f admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:44.220504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef493891-2adf-4d1a-b467-074f59201c8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:44.220504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef493891-2adf-4d1a-b467-074f59201c8f admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:44.220684 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618307fc-2001-4a73-8192-5f0da2562d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:44.220684 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618307fc-2001-4a73-8192-5f0da2562d18 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:44.220684 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618307fc-2001-4a73-8192-5f0da2562d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:44.220684 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618307fc-2001-4a73-8192-5f0da2562d18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:44.221001 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618307fc-2001-4a73-8192-5f0da2562d18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:44.221001 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618307fc-2001-4a73-8192-5f0da2562d18 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:45.548328 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 934/1863] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:24:44 2022] POST /identity/v3/users => generated 294 bytes in 1465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:45.548328 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef493891-2adf-4d1a-b467-074f59201c8f admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:45.548985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 930/1864] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:24:44 2022] POST /identity/v3/projects => generated 407 bytes in 1466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:45.550858 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618307fc-2001-4a73-8192-5f0da2562d18 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:45.551196 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-ef493891-2adf-4d1a-b467-074f59201c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:45.633127 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef493891-2adf-4d1a-b467-074f59201c8f admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:45.634240 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618307fc-2001-4a73-8192-5f0da2562d18 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:45.640134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1927450-c583-4c8b-9dc7-42014d623f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:45.641083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1927450-c583-4c8b-9dc7-42014d623f2c admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:45.641512 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1927450-c583-4c8b-9dc7-42014d623f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:45.641922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1927450-c583-4c8b-9dc7-42014d623f2c admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:45.643824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1927450-c583-4c8b-9dc7-42014d623f2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:45.646225 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1927450-c583-4c8b-9dc7-42014d623f2c admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:45.686353 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac764c4-c5ac-4512-a8bd-a0d4a352a60a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:45.686905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac764c4-c5ac-4512-a8bd-a0d4a352a60a service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:45.687233 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac764c4-c5ac-4512-a8bd-a0d4a352a60a service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:45.687543 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac764c4-c5ac-4512-a8bd-a0d4a352a60a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:45.689295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-8ac764c4-c5ac-4512-a8bd-a0d4a352a60a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:45.752592 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac764c4-c5ac-4512-a8bd-a0d4a352a60a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:45.754293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac764c4-c5ac-4512-a8bd-a0d4a352a60a service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:45.764036 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 935/1865] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:45 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:45.764888 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac764c4-c5ac-4512-a8bd-a0d4a352a60a service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:45.774826 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac764c4-c5ac-4512-a8bd-a0d4a352a60a service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:45.782264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046df91-9a80-457a-ae7c-03652561e3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:45.783213 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046df91-9a80-457a-ae7c-03652561e3e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:45.783505 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046df91-9a80-457a-ae7c-03652561e3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:45.783787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046df91-9a80-457a-ae7c-03652561e3e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:45.784583 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5046df91-9a80-457a-ae7c-03652561e3e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:45.786211 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 931/1866] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:24:45 2022] POST /identity/v3/users => generated 304 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:24:45.786356 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5046df91-9a80-457a-ae7c-03652561e3e2 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:45.790032 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1927450-c583-4c8b-9dc7-42014d623f2c admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:45.803323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 936/1867] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:45.805427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1927450-c583-4c8b-9dc7-42014d623f2c admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:45.808476 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046df91-9a80-457a-ae7c-03652561e3e2 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:45.817043 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc1284-3913-478b-ac35-46a0745925e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:45.817945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc1284-3913-478b-ac35-46a0745925e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:45.818346 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc1284-3913-478b-ac35-46a0745925e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:45.818808 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc1284-3913-478b-ac35-46a0745925e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:45.819570 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046df91-9a80-457a-ae7c-03652561e3e2 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:45.819698 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc1284-3913-478b-ac35-46a0745925e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:45.821620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc1284-3913-478b-ac35-46a0745925e7 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:45.824307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd03ee9-1165-42ea-b552-5f651e2c2186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:45.824982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd03ee9-1165-42ea-b552-5f651e2c2186 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:45.825273 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd03ee9-1165-42ea-b552-5f651e2c2186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:45.825582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd03ee9-1165-42ea-b552-5f651e2c2186 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:45.826315 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd03ee9-1165-42ea-b552-5f651e2c2186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:45.827979 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd03ee9-1165-42ea-b552-5f651e2c2186 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:45.836947 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 932/1868] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:24:45.839798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 937/1869] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:24:45.839945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc1284-3913-478b-ac35-46a0745925e7 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:45.844418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd03ee9-1165-42ea-b552-5f651e2c2186 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:45.851725 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc1284-3913-478b-ac35-46a0745925e7 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:45.856981 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd03ee9-1165-42ea-b552-5f651e2c2186 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:45.861221 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89560c75-756f-47b1-a6c2-a5a120383822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:45.861221 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89560c75-756f-47b1-a6c2-a5a120383822 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:45.861221 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89560c75-756f-47b1-a6c2-a5a120383822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:45.861221 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89560c75-756f-47b1-a6c2-a5a120383822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:45.861562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89560c75-756f-47b1-a6c2-a5a120383822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:45.861817 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89560c75-756f-47b1-a6c2-a5a120383822 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:45.863548 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d20c41-ffa1-44da-b4f7-7d59411c71ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:45.864554 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d20c41-ffa1-44da-b4f7-7d59411c71ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:45.864927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d20c41-ffa1-44da-b4f7-7d59411c71ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:45.865280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d20c41-ffa1-44da-b4f7-7d59411c71ea admin admin] PATH_INFO: `/v3/projects/ae17a090e1464c58a9d4563a1109c7bd/users/477600651b1b4197b78dfa7d073f5ac5/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:45.873744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 933/1870] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:45.877852 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89560c75-756f-47b1-a6c2-a5a120383822 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:45.883482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d20c41-ffa1-44da-b4f7-7d59411c71ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae17a090e1464c58a9d4563a1109c7bd, user_id=477600651b1b4197b78dfa7d073f5ac5, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:45.886470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d20c41-ffa1-44da-b4f7-7d59411c71ea admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:45.897859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89560c75-756f-47b1-a6c2-a5a120383822 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:45.902887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d24dcd-29d7-4197-a0f5-a8cfef5349c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:45.904415 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d24dcd-29d7-4197-a0f5-a8cfef5349c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:45.904879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d24dcd-29d7-4197-a0f5-a8cfef5349c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:45.905642 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d24dcd-29d7-4197-a0f5-a8cfef5349c4 admin admin] PATH_INFO: `/v3/projects/0034c51704ca48ef90af2caa66968099/users/c85838cd819d41da8978dfeebe370a04/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:45.931874 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d24dcd-29d7-4197-a0f5-a8cfef5349c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0034c51704ca48ef90af2caa66968099, user_id=c85838cd819d41da8978dfeebe370a04, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:45.934734 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d24dcd-29d7-4197-a0f5-a8cfef5349c4 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:46.008271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 938/1871] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:45 2022] PUT /identity/v3/projects/ae17a090e1464c58a9d4563a1109c7bd/users/477600651b1b4197b78dfa7d073f5ac5/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:24:46.019876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc0b11-edf3-4fa3-b548-56e7f845e36d None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:46.020170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc0b11-edf3-4fa3-b548-56e7f845e36d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:46.020453 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc0b11-edf3-4fa3-b548-56e7f845e36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:46.035797 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 934/1872] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:45 2022] PUT /identity/v3/projects/0034c51704ca48ef90af2caa66968099/users/c85838cd819d41da8978dfeebe370a04/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:24:46.049869 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-4abc0b11-edf3-4fa3-b548-56e7f845e36d None None] MFA Rules not processed for user `477600651b1b4197b78dfa7d073f5ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:46.050840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-4abc0b11-edf3-4fa3-b548-56e7f845e36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:46.051286 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b34ac-36fa-4a25-9245-a4ab7b08ae55 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:46.052427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b34ac-36fa-4a25-9245-a4ab7b08ae55 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:46.052979 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b34ac-36fa-4a25-9245-a4ab7b08ae55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:46.082056 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-1f8b34ac-36fa-4a25-9245-a4ab7b08ae55 None None] MFA Rules not processed for user `c85838cd819d41da8978dfeebe370a04`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:46.083822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-1f8b34ac-36fa-4a25-9245-a4ab7b08ae55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:46.126494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 939/1873] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:24:46.130816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc0b11-edf3-4fa3-b548-56e7f845e36d None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:46.158590 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 935/1874] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:46 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:24:46.163623 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8b34ac-36fa-4a25-9245-a4ab7b08ae55 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:46.217268 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc0b11-edf3-4fa3-b548-56e7f845e36d None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:46.220769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:46.222598 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b62d72-3fe5-4c2c-9d37-8fc11f5b4af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:46.223358 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b62d72-3fe5-4c2c-9d37-8fc11f5b4af2 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:46.223591 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b62d72-3fe5-4c2c-9d37-8fc11f5b4af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:46.223869 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b62d72-3fe5-4c2c-9d37-8fc11f5b4af2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:46.224844 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b62d72-3fe5-4c2c-9d37-8fc11f5b4af2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:46.227931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b62d72-3fe5-4c2c-9d37-8fc11f5b4af2 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:46.243477 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8b34ac-36fa-4a25-9245-a4ab7b08ae55 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:46.250565 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41f1df-98bf-4552-8d3e-efa85e20d72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:46.251745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41f1df-98bf-4552-8d3e-efa85e20d72f admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:46.252835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41f1df-98bf-4552-8d3e-efa85e20d72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:46.253423 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41f1df-98bf-4552-8d3e-efa85e20d72f admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:46.254438 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41f1df-98bf-4552-8d3e-efa85e20d72f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:46.256917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41f1df-98bf-4552-8d3e-efa85e20d72f admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:46.339806 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 940/1875] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:24:46 2022] POST /identity/v3/projects => generated 383 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:24:46.344260 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b62d72-3fe5-4c2c-9d37-8fc11f5b4af2 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:46.373122 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 936/1876] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:24:46 2022] POST /identity/v3/projects => generated 407 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:24:46.378867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41f1df-98bf-4552-8d3e-efa85e20d72f admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:46.412997 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b62d72-3fe5-4c2c-9d37-8fc11f5b4af2 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:46.447047 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41f1df-98bf-4552-8d3e-efa85e20d72f admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:46.452309 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61fe87-d88f-457a-87c5-8daacfbc54d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:46.453543 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61fe87-d88f-457a-87c5-8daacfbc54d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:46.453844 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61fe87-d88f-457a-87c5-8daacfbc54d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:46.454128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61fe87-d88f-457a-87c5-8daacfbc54d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:46.456615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61fe87-d88f-457a-87c5-8daacfbc54d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:46.458818 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61fe87-d88f-457a-87c5-8daacfbc54d7 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:46.476137 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba083962-fc0f-493f-bdfe-b9cbe4aa9ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:46.476844 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba083962-fc0f-493f-bdfe-b9cbe4aa9ff9 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:46.477120 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba083962-fc0f-493f-bdfe-b9cbe4aa9ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:46.477393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba083962-fc0f-493f-bdfe-b9cbe4aa9ff9 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:46.478149 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba083962-fc0f-493f-bdfe-b9cbe4aa9ff9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:46.480238 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba083962-fc0f-493f-bdfe-b9cbe4aa9ff9 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:46.568091 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 937/1877] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:24:46 2022] POST /identity/v3/users => generated 310 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:46.572033 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61fe87-d88f-457a-87c5-8daacfbc54d7 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:46.582414 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61fe87-d88f-457a-87c5-8daacfbc54d7 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:46.590302 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 941/1878] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:24:46 2022] POST /identity/v3/users => generated 298 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:24:46.596739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894eece5-ec4b-4bbb-93c8-43a8eea81874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:46.597573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894eece5-ec4b-4bbb-93c8-43a8eea81874 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:46.597733 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894eece5-ec4b-4bbb-93c8-43a8eea81874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:46.598022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894eece5-ec4b-4bbb-93c8-43a8eea81874 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:46.598857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894eece5-ec4b-4bbb-93c8-43a8eea81874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:46.599202 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba083962-fc0f-493f-bdfe-b9cbe4aa9ff9 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:46.601489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894eece5-ec4b-4bbb-93c8-43a8eea81874 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:46.613806 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba083962-fc0f-493f-bdfe-b9cbe4aa9ff9 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:46.618438 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 938/1879] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:24:46.619790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894eece5-ec4b-4bbb-93c8-43a8eea81874 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:46.622972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9ced5-b094-4be8-b12e-757d5b9b62e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:46.623684 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9ced5-b094-4be8-b12e-757d5b9b62e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:46.630047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9ced5-b094-4be8-b12e-757d5b9b62e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:46.630047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9ced5-b094-4be8-b12e-757d5b9b62e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:46.630047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9ced5-b094-4be8-b12e-757d5b9b62e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:46.630047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9ced5-b094-4be8-b12e-757d5b9b62e8 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:46.632376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894eece5-ec4b-4bbb-93c8-43a8eea81874 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:46.642287 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 942/1880] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:46.644328 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf4450-635a-498a-aacb-c89397d94dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:46.645381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf4450-635a-498a-aacb-c89397d94dc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:46.645672 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf4450-635a-498a-aacb-c89397d94dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:46.645952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf4450-635a-498a-aacb-c89397d94dc0 admin admin] PATH_INFO: `/v3/projects/b8242a3bcd4444bbb162274ae10bd3b5/users/0bc0adf48e9a47d0a7b5c137a34a5916/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:46.647673 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9ced5-b094-4be8-b12e-757d5b9b62e8 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:46.661518 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9ced5-b094-4be8-b12e-757d5b9b62e8 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:46.672130 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf4450-635a-498a-aacb-c89397d94dc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8242a3bcd4444bbb162274ae10bd3b5, user_id=0bc0adf48e9a47d0a7b5c137a34a5916, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:46.685306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac483d7-e9da-42bf-abfd-8eae4b22f589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:46.685306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac483d7-e9da-42bf-abfd-8eae4b22f589 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:46.685306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac483d7-e9da-42bf-abfd-8eae4b22f589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:46.685306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac483d7-e9da-42bf-abfd-8eae4b22f589 admin admin] PATH_INFO: `/v3/projects/dea5d163d2c94a9e80e7aac2ff1f0a00/users/12796d7bef11495bb80ed4d3fa17fd8c/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:46.685647 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf4450-635a-498a-aacb-c89397d94dc0 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:46.696430 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac483d7-e9da-42bf-abfd-8eae4b22f589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dea5d163d2c94a9e80e7aac2ff1f0a00, user_id=12796d7bef11495bb80ed4d3fa17fd8c, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:46.699271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac483d7-e9da-42bf-abfd-8eae4b22f589 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:46.847532 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 939/1881] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:46 2022] PUT /identity/v3/projects/b8242a3bcd4444bbb162274ae10bd3b5/users/0bc0adf48e9a47d0a7b5c137a34a5916/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:46.851420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf4450-635a-498a-aacb-c89397d94dc0 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:46.912607 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf4450-635a-498a-aacb-c89397d94dc0 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:46.917872 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006692e-648f-49a7-aba3-9b31e23b37a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:46.918543 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006692e-648f-49a7-aba3-9b31e23b37a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:46.918833 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006692e-648f-49a7-aba3-9b31e23b37a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:46.919142 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006692e-648f-49a7-aba3-9b31e23b37a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:46.920917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006692e-648f-49a7-aba3-9b31e23b37a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:46.922592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006692e-648f-49a7-aba3-9b31e23b37a4 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:46.923406 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:46.929613 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 943/1882] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:46 2022] PUT /identity/v3/projects/dea5d163d2c94a9e80e7aac2ff1f0a00/users/12796d7bef11495bb80ed4d3fa17fd8c/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:46.933684 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac483d7-e9da-42bf-abfd-8eae4b22f589 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:46.936838 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 940/1883] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:24:46.937874 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006692e-648f-49a7-aba3-9b31e23b37a4 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:46.948756 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006692e-648f-49a7-aba3-9b31e23b37a4 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:47.018773 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0c882-84a6-4b46-b454-5ae92d39a561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:47.019718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0c882-84a6-4b46-b454-5ae92d39a561 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:47.019991 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0c882-84a6-4b46-b454-5ae92d39a561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:47.020279 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0c882-84a6-4b46-b454-5ae92d39a561 admin admin] PATH_INFO: `/v3/projects/b8242a3bcd4444bbb162274ae10bd3b5/users/0bc0adf48e9a47d0a7b5c137a34a5916/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:47.024660 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0c882-84a6-4b46-b454-5ae92d39a561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8242a3bcd4444bbb162274ae10bd3b5, user_id=0bc0adf48e9a47d0a7b5c137a34a5916, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:47.027791 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0c882-84a6-4b46-b454-5ae92d39a561 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:47.033075 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac483d7-e9da-42bf-abfd-8eae4b22f589 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:47.040580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdda153-5793-43ef-9f6e-ec26327d0dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:47.040580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdda153-5793-43ef-9f6e-ec26327d0dac admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:47.040580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdda153-5793-43ef-9f6e-ec26327d0dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:47.040580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdda153-5793-43ef-9f6e-ec26327d0dac admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:47.041014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdda153-5793-43ef-9f6e-ec26327d0dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:47.041847 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdda153-5793-43ef-9f6e-ec26327d0dac admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:47.042372 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 941/1884] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:46 2022] PUT /identity/v3/projects/b8242a3bcd4444bbb162274ae10bd3b5/users/0bc0adf48e9a47d0a7b5c137a34a5916/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:47.055429 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 944/1885] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:46 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:24:47.055913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515687a-979f-4c5d-a14e-31902f45db83 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:47.055913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515687a-979f-4c5d-a14e-31902f45db83 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:47.055913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515687a-979f-4c5d-a14e-31902f45db83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:47.056848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdda153-5793-43ef-9f6e-ec26327d0dac admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:47.068167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdda153-5793-43ef-9f6e-ec26327d0dac admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:47.085831 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-3515687a-979f-4c5d-a14e-31902f45db83 None None] MFA Rules not processed for user `0bc0adf48e9a47d0a7b5c137a34a5916`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:47.087251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-3515687a-979f-4c5d-a14e-31902f45db83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:47.134247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77356293-f0be-4dd4-b4e4-61d481659474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:47.135277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77356293-f0be-4dd4-b4e4-61d481659474 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:47.135764 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77356293-f0be-4dd4-b4e4-61d481659474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:47.135894 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77356293-f0be-4dd4-b4e4-61d481659474 admin admin] PATH_INFO: `/v3/projects/dea5d163d2c94a9e80e7aac2ff1f0a00/users/12796d7bef11495bb80ed4d3fa17fd8c/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:47.139261 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77356293-f0be-4dd4-b4e4-61d481659474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dea5d163d2c94a9e80e7aac2ff1f0a00, user_id=12796d7bef11495bb80ed4d3fa17fd8c, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:47.141359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77356293-f0be-4dd4-b4e4-61d481659474 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:47.159715 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 945/1886] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:47 2022] PUT /identity/v3/projects/dea5d163d2c94a9e80e7aac2ff1f0a00/users/12796d7bef11495bb80ed4d3fa17fd8c/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:47.163457 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 942/1887] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:47 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:24:47.176738 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50500dab-f630-42d2-a234-68963d96bc59 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:47.177165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50500dab-f630-42d2-a234-68963d96bc59 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:47.177459 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50500dab-f630-42d2-a234-68963d96bc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:47.179091 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515687a-979f-4c5d-a14e-31902f45db83 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:47.209371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-50500dab-f630-42d2-a234-68963d96bc59 None None] MFA Rules not processed for user `12796d7bef11495bb80ed4d3fa17fd8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:47.211092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-50500dab-f630-42d2-a234-68963d96bc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:47.246228 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515687a-979f-4c5d-a14e-31902f45db83 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:47.291104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 946/1888] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:47.302563 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50500dab-f630-42d2-a234-68963d96bc59 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:47.303980 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-50500dab-f630-42d2-a234-68963d96bc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:47.310538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9502a2-78a9-4926-bd2b-4de430e9e48b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:47.311315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9502a2-78a9-4926-bd2b-4de430e9e48b service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:47.311672 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9502a2-78a9-4926-bd2b-4de430e9e48b service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:47.312060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9502a2-78a9-4926-bd2b-4de430e9e48b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:47.370682 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50500dab-f630-42d2-a234-68963d96bc59 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:47.379249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a5fd8-1e6a-4c15-af15-d259bf81cf0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:47.379939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a5fd8-1e6a-4c15-af15-d259bf81cf0e service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:47.380369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a5fd8-1e6a-4c15-af15-d259bf81cf0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:47.380807 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a5fd8-1e6a-4c15-af15-d259bf81cf0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:47.385228 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9502a2-78a9-4926-bd2b-4de430e9e48b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:47.386801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9502a2-78a9-4926-bd2b-4de430e9e48b service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:47.391174 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a5fd8-1e6a-4c15-af15-d259bf81cf0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:47.392759 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a5fd8-1e6a-4c15-af15-d259bf81cf0e service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:47.403026 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 943/1889] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:47 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:24:47.404170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 947/1890] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:24:47.663913 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:47.970414 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9502a2-78a9-4926-bd2b-4de430e9e48b service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:47.979488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9502a2-78a9-4926-bd2b-4de430e9e48b service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:47.984213 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67870f30-39d4-4978-9ed9-b145deee679d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:47.985378 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67870f30-39d4-4978-9ed9-b145deee679d service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:47.985642 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67870f30-39d4-4978-9ed9-b145deee679d service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:47.985916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67870f30-39d4-4978-9ed9-b145deee679d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:48.044125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67870f30-39d4-4978-9ed9-b145deee679d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:48.045603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67870f30-39d4-4978-9ed9-b145deee679d service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:48.058201 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 944/1891] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:47 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:48.766190 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a5fd8-1e6a-4c15-af15-d259bf81cf0e service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:48.833400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a5fd8-1e6a-4c15-af15-d259bf81cf0e service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:48.839809 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d432c29-b610-4340-b28f-13b4202dbd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:48.840715 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d432c29-b610-4340-b28f-13b4202dbd7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:48.840999 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d432c29-b610-4340-b28f-13b4202dbd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:48.841279 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d432c29-b610-4340-b28f-13b4202dbd7f admin admin] PATH_INFO: `/v3/users/477600651b1b4197b78dfa7d073f5ac5` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:48.842505 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d432c29-b610-4340-b28f-13b4202dbd7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=477600651b1b4197b78dfa7d073f5ac5)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:48.844538 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d432c29-b610-4340-b28f-13b4202dbd7f admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:48.926838 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:49.119484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-1d432c29-b610-4340-b28f-13b4202dbd7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '477600651b1b4197b78dfa7d073f5ac5'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:49.131311 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-1d432c29-b610-4340-b28f-13b4202dbd7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '477600651b1b4197b78dfa7d073f5ac5'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:49.132048 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-1d432c29-b610-4340-b28f-13b4202dbd7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '477600651b1b4197b78dfa7d073f5ac5'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:49.132665 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-1d432c29-b610-4340-b28f-13b4202dbd7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '477600651b1b4197b78dfa7d073f5ac5'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:49.145887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-1d432c29-b610-4340-b28f-13b4202dbd7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '477600651b1b4197b78dfa7d073f5ac5'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:49.298141 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 948/1892] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:24:48 2022] DELETE /identity/v3/users/477600651b1b4197b78dfa7d073f5ac5 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:49.302692 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ef695-1149-4e9c-9c10-d6551fcc8778 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:49.303184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ef695-1149-4e9c-9c10-d6551fcc8778 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:49.303411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ef695-1149-4e9c-9c10-d6551fcc8778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:49.323052 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-0e3ef695-1149-4e9c-9c10-d6551fcc8778 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:49.324663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-0e3ef695-1149-4e9c-9c10-d6551fcc8778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:49.385451 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 945/1893] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:49.394892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d432c29-b610-4340-b28f-13b4202dbd7f admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:49.396306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-1d432c29-b610-4340-b28f-13b4202dbd7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:49.452887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d432c29-b610-4340-b28f-13b4202dbd7f admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:49.456542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fef190-58a9-441e-ae8b-7def992548cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:49.457191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fef190-58a9-441e-ae8b-7def992548cb service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:49.457476 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fef190-58a9-441e-ae8b-7def992548cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:49.457753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fef190-58a9-441e-ae8b-7def992548cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:49.466992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fef190-58a9-441e-ae8b-7def992548cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:49.469275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fef190-58a9-441e-ae8b-7def992548cb service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:49.478839 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 949/1894] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:49.743844 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:49.806929 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:49.882278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ef695-1149-4e9c-9c10-d6551fcc8778 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:49.883709 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-0e3ef695-1149-4e9c-9c10-d6551fcc8778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:49.897998 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ef695-1149-4e9c-9c10-d6551fcc8778 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:49.905396 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a282bd-f7bf-4112-8f36-5c2f082e505f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:49.907140 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a282bd-f7bf-4112-8f36-5c2f082e505f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:49.907442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a282bd-f7bf-4112-8f36-5c2f082e505f admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:49.908245 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a282bd-f7bf-4112-8f36-5c2f082e505f admin admin] PATH_INFO: `/v3/projects/ae17a090e1464c58a9d4563a1109c7bd` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:49.909402 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a282bd-f7bf-4112-8f36-5c2f082e505f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae17a090e1464c58a9d4563a1109c7bd)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:49.911383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a282bd-f7bf-4112-8f36-5c2f082e505f admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:50.040964 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-71a282bd-f7bf-4112-8f36-5c2f082e505f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae17a090e1464c58a9d4563a1109c7bd', 'request_id': 'req-71a282bd-f7bf-4112-8f36-5c2f082e505f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:50.047705 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-71a282bd-f7bf-4112-8f36-5c2f082e505f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae17a090e1464c58a9d4563a1109c7bd', 'request_id': 'req-71a282bd-f7bf-4112-8f36-5c2f082e505f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:50.120758 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-71a282bd-f7bf-4112-8f36-5c2f082e505f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae17a090e1464c58a9d4563a1109c7bd', 'request_id': 'req-71a282bd-f7bf-4112-8f36-5c2f082e505f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:50.132707 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-71a282bd-f7bf-4112-8f36-5c2f082e505f admin admin] The token cache is being invalidate because project ae17a090e1464c58a9d4563a1109c7bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:24:50.132928 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-71a282bd-f7bf-4112-8f36-5c2f082e505f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:50.133610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-71a282bd-f7bf-4112-8f36-5c2f082e505f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:50.135211 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 946/1895] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:24:49 2022] DELETE /identity/v3/projects/ae17a090e1464c58a9d4563a1109c7bd => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:50.139565 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fef190-58a9-441e-ae8b-7def992548cb service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:50.151300 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fef190-58a9-441e-ae8b-7def992548cb service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:50.222830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe14bbf-bb94-41fc-a427-9fae7cf0c60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:50.223746 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe14bbf-bb94-41fc-a427-9fae7cf0c60c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:50.224031 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe14bbf-bb94-41fc-a427-9fae7cf0c60c admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:50.224306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe14bbf-bb94-41fc-a427-9fae7cf0c60c admin admin] PATH_INFO: `/v3/users/12796d7bef11495bb80ed4d3fa17fd8c` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:50.225495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe14bbf-bb94-41fc-a427-9fae7cf0c60c admin admin] RBAC: Authorizing `identity:delete_user(user_id=12796d7bef11495bb80ed4d3fa17fd8c)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:50.227625 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe14bbf-bb94-41fc-a427-9fae7cf0c60c admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:50.449364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ffe14bbf-bb94-41fc-a427-9fae7cf0c60c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12796d7bef11495bb80ed4d3fa17fd8c'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:50.456837 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ffe14bbf-bb94-41fc-a427-9fae7cf0c60c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12796d7bef11495bb80ed4d3fa17fd8c'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:50.457530 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ffe14bbf-bb94-41fc-a427-9fae7cf0c60c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12796d7bef11495bb80ed4d3fa17fd8c'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:50.458140 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ffe14bbf-bb94-41fc-a427-9fae7cf0c60c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12796d7bef11495bb80ed4d3fa17fd8c'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:50.469696 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ffe14bbf-bb94-41fc-a427-9fae7cf0c60c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12796d7bef11495bb80ed4d3fa17fd8c'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:50.544094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 950/1896] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:24:50 2022] DELETE /identity/v3/users/12796d7bef11495bb80ed4d3fa17fd8c => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:50.892882 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:51.065976 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a282bd-f7bf-4112-8f36-5c2f082e505f admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:51.069346 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-71a282bd-f7bf-4112-8f36-5c2f082e505f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:51.093045 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a282bd-f7bf-4112-8f36-5c2f082e505f admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:51.150748 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460ee58-d695-49e9-968a-fcbd5c857a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:51.151648 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460ee58-d695-49e9-968a-fcbd5c857a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:51.151930 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460ee58-d695-49e9-968a-fcbd5c857a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:51.152207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460ee58-d695-49e9-968a-fcbd5c857a15 admin admin] PATH_INFO: `/v3/projects/dea5d163d2c94a9e80e7aac2ff1f0a00` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:51.153412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d460ee58-d695-49e9-968a-fcbd5c857a15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dea5d163d2c94a9e80e7aac2ff1f0a00)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:51.155447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d460ee58-d695-49e9-968a-fcbd5c857a15 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:51.328257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d460ee58-d695-49e9-968a-fcbd5c857a15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dea5d163d2c94a9e80e7aac2ff1f0a00', 'request_id': 'req-d460ee58-d695-49e9-968a-fcbd5c857a15'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:51.336635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d460ee58-d695-49e9-968a-fcbd5c857a15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dea5d163d2c94a9e80e7aac2ff1f0a00', 'request_id': 'req-d460ee58-d695-49e9-968a-fcbd5c857a15'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:51.395393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:51.423707 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d460ee58-d695-49e9-968a-fcbd5c857a15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dea5d163d2c94a9e80e7aac2ff1f0a00', 'request_id': 'req-d460ee58-d695-49e9-968a-fcbd5c857a15'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:51.435896 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d460ee58-d695-49e9-968a-fcbd5c857a15 admin admin] The token cache is being invalidate because project dea5d163d2c94a9e80e7aac2ff1f0a00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:24:51.436194 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d460ee58-d695-49e9-968a-fcbd5c857a15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:51.436846 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d460ee58-d695-49e9-968a-fcbd5c857a15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:24:51.438517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 947/1897] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:24:51 2022] DELETE /identity/v3/projects/dea5d163d2c94a9e80e7aac2ff1f0a00 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:51.443393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032830dd-97c0-4f6f-b354-b41d38049327 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:51.443693 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032830dd-97c0-4f6f-b354-b41d38049327 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:51.443975 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032830dd-97c0-4f6f-b354-b41d38049327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:51.463055 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-032830dd-97c0-4f6f-b354-b41d38049327 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:51.464295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-032830dd-97c0-4f6f-b354-b41d38049327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:51.536919 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 951/1898] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:24:51.544560 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43af77-5f8d-40c2-baed-88584f0b773a None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:51.544860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43af77-5f8d-40c2-baed-88584f0b773a None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:51.545147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43af77-5f8d-40c2-baed-88584f0b773a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:51.564443 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-ce43af77-5f8d-40c2-baed-88584f0b773a None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:51.565678 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ce43af77-5f8d-40c2-baed-88584f0b773a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:51.575358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 948/1899] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:51.594593 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032830dd-97c0-4f6f-b354-b41d38049327 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:51.605599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032830dd-97c0-4f6f-b354-b41d38049327 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:51.611004 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a853e65-c6dc-4860-abc1-67b3977ececf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:51.611681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a853e65-c6dc-4860-abc1-67b3977ececf admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:51.611970 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a853e65-c6dc-4860-abc1-67b3977ececf admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:51.612294 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a853e65-c6dc-4860-abc1-67b3977ececf admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:51.613028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a853e65-c6dc-4860-abc1-67b3977ececf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:24:51.613501 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a853e65-c6dc-4860-abc1-67b3977ececf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:51.615143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a853e65-c6dc-4860-abc1-67b3977ececf admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:51.628129 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 952/1900] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:24:51.632502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43af77-5f8d-40c2-baed-88584f0b773a None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:51.647424 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43af77-5f8d-40c2-baed-88584f0b773a None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:51.652729 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3b63f-1970-40c5-8ccc-9d984aa91db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:51.656354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3b63f-1970-40c5-8ccc-9d984aa91db8 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:51.656354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3b63f-1970-40c5-8ccc-9d984aa91db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:51.656354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3b63f-1970-40c5-8ccc-9d984aa91db8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:51.656354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af3b63f-1970-40c5-8ccc-9d984aa91db8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:51.656735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af3b63f-1970-40c5-8ccc-9d984aa91db8 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:51.774485 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 949/1901] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:24:51 2022] POST /identity/v3/projects => generated 375 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:51.778822 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a853e65-c6dc-4860-abc1-67b3977ececf admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:51.788838 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a853e65-c6dc-4860-abc1-67b3977ececf admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:51.845672 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924fcfc-1fe5-4e43-ac02-7af33d5c83aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:51.846361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924fcfc-1fe5-4e43-ac02-7af33d5c83aa admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:51.846654 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924fcfc-1fe5-4e43-ac02-7af33d5c83aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:51.846960 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924fcfc-1fe5-4e43-ac02-7af33d5c83aa admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:51.847674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9924fcfc-1fe5-4e43-ac02-7af33d5c83aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:51.849602 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9924fcfc-1fe5-4e43-ac02-7af33d5c83aa admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:51.964469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 953/1902] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:24:51 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:51.991682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3b63f-1970-40c5-8ccc-9d984aa91db8 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:51.991682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3b63f-1970-40c5-8ccc-9d984aa91db8 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:51.991682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667edfb-1267-4b7a-b31e-5e38153be666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:51.991682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667edfb-1267-4b7a-b31e-5e38153be666 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:51.992402 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667edfb-1267-4b7a-b31e-5e38153be666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:51.992402 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667edfb-1267-4b7a-b31e-5e38153be666 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:51.992402 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667edfb-1267-4b7a-b31e-5e38153be666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:51.992402 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667edfb-1267-4b7a-b31e-5e38153be666 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:52.000379 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 950/1903] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:24:52.004284 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924fcfc-1fe5-4e43-ac02-7af33d5c83aa admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:52.014509 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924fcfc-1fe5-4e43-ac02-7af33d5c83aa admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:52.019295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c53d15-5dc1-40ed-92dd-ea519004a9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:52.020071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c53d15-5dc1-40ed-92dd-ea519004a9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:52.020241 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c53d15-5dc1-40ed-92dd-ea519004a9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:52.020520 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c53d15-5dc1-40ed-92dd-ea519004a9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:52.021249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c53d15-5dc1-40ed-92dd-ea519004a9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:52.022840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c53d15-5dc1-40ed-92dd-ea519004a9a6 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:52.038279 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 954/1904] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:52.043345 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667edfb-1267-4b7a-b31e-5e38153be666 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:52.054375 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667edfb-1267-4b7a-b31e-5e38153be666 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:52.059258 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414dbff-4394-4ccb-a457-9c0b6eff7968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:52.060160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414dbff-4394-4ccb-a457-9c0b6eff7968 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:52.060472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414dbff-4394-4ccb-a457-9c0b6eff7968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:52.060755 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414dbff-4394-4ccb-a457-9c0b6eff7968 admin admin] PATH_INFO: `/v3/projects/41e68f66f1674c44a68c89df35df0609/users/e536b7083a964148a5bdffe90aada4be/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:52.078326 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414dbff-4394-4ccb-a457-9c0b6eff7968 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41e68f66f1674c44a68c89df35df0609, user_id=e536b7083a964148a5bdffe90aada4be, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:52.081106 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414dbff-4394-4ccb-a457-9c0b6eff7968 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:52.160948 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 951/1905] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:52 2022] PUT /identity/v3/projects/41e68f66f1674c44a68c89df35df0609/users/e536b7083a964148a5bdffe90aada4be/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:52.172627 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeec514-0f6d-478f-be3c-08fb5d10580d None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:52.173036 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeec514-0f6d-478f-be3c-08fb5d10580d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:52.173399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeec514-0f6d-478f-be3c-08fb5d10580d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:52.210034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-bfeec514-0f6d-478f-be3c-08fb5d10580d None None] MFA Rules not processed for user `e536b7083a964148a5bdffe90aada4be`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:52.211297 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-bfeec514-0f6d-478f-be3c-08fb5d10580d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:52.275271 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:52.284151 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 955/1906] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:52.289111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414dbff-4394-4ccb-a457-9c0b6eff7968 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:52.353169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414dbff-4394-4ccb-a457-9c0b6eff7968 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:52.358296 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c00ab1-8c9f-4320-b19b-b80e365eeb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:52.359041 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c00ab1-8c9f-4320-b19b-b80e365eeb9d admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:52.360264 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c00ab1-8c9f-4320-b19b-b80e365eeb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:52.360525 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c00ab1-8c9f-4320-b19b-b80e365eeb9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:52.361622 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c00ab1-8c9f-4320-b19b-b80e365eeb9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:52.365121 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c00ab1-8c9f-4320-b19b-b80e365eeb9d admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:52.505629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 952/1907] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:24:52 2022] POST /identity/v3/projects => generated 377 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:52.510695 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeec514-0f6d-478f-be3c-08fb5d10580d None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:52.539700 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeec514-0f6d-478f-be3c-08fb5d10580d None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:52.643295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e989724-c342-40a2-9e45-77fb4d5b7362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:52.644393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e989724-c342-40a2-9e45-77fb4d5b7362 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:52.645044 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e989724-c342-40a2-9e45-77fb4d5b7362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:52.649813 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e989724-c342-40a2-9e45-77fb4d5b7362 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:52.651115 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e989724-c342-40a2-9e45-77fb4d5b7362 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:52.653504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e989724-c342-40a2-9e45-77fb4d5b7362 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:52.811868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 956/1908] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:24:52 2022] POST /identity/v3/users => generated 289 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:52.815058 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c00ab1-8c9f-4320-b19b-b80e365eeb9d admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:52.842818 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c00ab1-8c9f-4320-b19b-b80e365eeb9d admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:52.842818 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb959d-2de6-4bf9-b8f3-b3c15f5a7ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:52.842818 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb959d-2de6-4bf9-b8f3-b3c15f5a7ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:52.842818 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb959d-2de6-4bf9-b8f3-b3c15f5a7ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:52.847229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb959d-2de6-4bf9-b8f3-b3c15f5a7ec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:52.847229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb959d-2de6-4bf9-b8f3-b3c15f5a7ec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:52.847229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb959d-2de6-4bf9-b8f3-b3c15f5a7ec1 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:52.854135 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 953/1909] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:52.858491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e989724-c342-40a2-9e45-77fb4d5b7362 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:52.869397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e989724-c342-40a2-9e45-77fb4d5b7362 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:52.875526 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa2326-1907-4413-85c4-d11b9cd9e248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:52.876207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa2326-1907-4413-85c4-d11b9cd9e248 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:52.876490 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa2326-1907-4413-85c4-d11b9cd9e248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:52.876769 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa2326-1907-4413-85c4-d11b9cd9e248 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:52.877505 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa2326-1907-4413-85c4-d11b9cd9e248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:52.879148 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa2326-1907-4413-85c4-d11b9cd9e248 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:52.890419 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 957/1910] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:52.894561 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb959d-2de6-4bf9-b8f3-b3c15f5a7ec1 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:52.904644 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb959d-2de6-4bf9-b8f3-b3c15f5a7ec1 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:52.909334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d9890-8a64-4b67-b7b0-3964d9258231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:52.910210 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d9890-8a64-4b67-b7b0-3964d9258231 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:52.910489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d9890-8a64-4b67-b7b0-3964d9258231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:52.910758 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d9890-8a64-4b67-b7b0-3964d9258231 admin admin] PATH_INFO: `/v3/projects/c4b794e3f4324a22be66224eafad3be6/users/78c278badeb248c394f5c1ebc9444fee/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:52.926693 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d9890-8a64-4b67-b7b0-3964d9258231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4b794e3f4324a22be66224eafad3be6, user_id=78c278badeb248c394f5c1ebc9444fee, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:52.933189 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d9890-8a64-4b67-b7b0-3964d9258231 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:53.069820 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 954/1911] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:52 2022] PUT /identity/v3/projects/c4b794e3f4324a22be66224eafad3be6/users/78c278badeb248c394f5c1ebc9444fee/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:24:53.079968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f10d2a-c996-4380-8dc9-3751349ce493 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:53.080266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f10d2a-c996-4380-8dc9-3751349ce493 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:53.080751 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f10d2a-c996-4380-8dc9-3751349ce493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:53.110401 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-25f10d2a-c996-4380-8dc9-3751349ce493 None None] MFA Rules not processed for user `78c278badeb248c394f5c1ebc9444fee`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:53.111660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-25f10d2a-c996-4380-8dc9-3751349ce493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:53.176364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 958/1912] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:24:53.180574 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d9890-8a64-4b67-b7b0-3964d9258231 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:53.247638 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d9890-8a64-4b67-b7b0-3964d9258231 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:53.252450 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c6cc0-5b17-4e34-bdaf-b42a147205bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:53.253111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c6cc0-5b17-4e34-bdaf-b42a147205bb admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:53.253401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c6cc0-5b17-4e34-bdaf-b42a147205bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:53.253681 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c6cc0-5b17-4e34-bdaf-b42a147205bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:53.254370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c6cc0-5b17-4e34-bdaf-b42a147205bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:53.256397 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c6cc0-5b17-4e34-bdaf-b42a147205bb admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:53.396463 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 955/1913] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:24:53 2022] POST /identity/v3/projects => generated 377 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:53.400618 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f10d2a-c996-4380-8dc9-3751349ce493 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:53.411851 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f10d2a-c996-4380-8dc9-3751349ce493 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:53.426018 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:53.483406 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c91db0-ad60-4d74-8e76-571c0a37bfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:53.484097 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c91db0-ad60-4d74-8e76-571c0a37bfaf admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:53.484387 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c91db0-ad60-4d74-8e76-571c0a37bfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:53.490191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c91db0-ad60-4d74-8e76-571c0a37bfaf admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:53.490191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c91db0-ad60-4d74-8e76-571c0a37bfaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:53.490191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c91db0-ad60-4d74-8e76-571c0a37bfaf admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:53.627457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 959/1914] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:24:53 2022] POST /identity/v3/users => generated 295 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:24:53.630194 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c6cc0-5b17-4e34-bdaf-b42a147205bb admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:53.641589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c6cc0-5b17-4e34-bdaf-b42a147205bb admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:53.646714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:53.648404 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c478f-04e3-4641-a0ca-0046e7eb9d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:53.649084 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c478f-04e3-4641-a0ca-0046e7eb9d69 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:53.649373 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c478f-04e3-4641-a0ca-0046e7eb9d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:53.649655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c478f-04e3-4641-a0ca-0046e7eb9d69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:53.650380 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c478f-04e3-4641-a0ca-0046e7eb9d69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:53.652030 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c478f-04e3-4641-a0ca-0046e7eb9d69 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:53.663837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 956/1915] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:53.668050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c91db0-ad60-4d74-8e76-571c0a37bfaf admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:53.678252 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c91db0-ad60-4d74-8e76-571c0a37bfaf admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:53.683242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae71f6b-5d87-448f-b199-c826ca0c8a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:53.684438 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae71f6b-5d87-448f-b199-c826ca0c8a6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:53.684844 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae71f6b-5d87-448f-b199-c826ca0c8a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:53.685125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae71f6b-5d87-448f-b199-c826ca0c8a6f admin admin] PATH_INFO: `/v3/projects/67b06f7f908c4ae19e6955f4c50e7ceb/users/5c5293e7fd8b49638b683090ff024a3f/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:53.700816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae71f6b-5d87-448f-b199-c826ca0c8a6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b06f7f908c4ae19e6955f4c50e7ceb, user_id=5c5293e7fd8b49638b683090ff024a3f, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:53.704111 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae71f6b-5d87-448f-b199-c826ca0c8a6f admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:53.789676 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 960/1916] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:53 2022] PUT /identity/v3/projects/67b06f7f908c4ae19e6955f4c50e7ceb/users/5c5293e7fd8b49638b683090ff024a3f/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:53.793584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c478f-04e3-4641-a0ca-0046e7eb9d69 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:53.803526 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c478f-04e3-4641-a0ca-0046e7eb9d69 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:53.862775 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27635478-51f5-4898-bb29-5916ec688550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:53.862775 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27635478-51f5-4898-bb29-5916ec688550 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:53.862775 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27635478-51f5-4898-bb29-5916ec688550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:53.862775 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27635478-51f5-4898-bb29-5916ec688550 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:53.863107 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27635478-51f5-4898-bb29-5916ec688550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:53.863548 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27635478-51f5-4898-bb29-5916ec688550 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:53.873409 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 957/1917] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:24:53.878134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae71f6b-5d87-448f-b199-c826ca0c8a6f admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:53.888560 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae71f6b-5d87-448f-b199-c826ca0c8a6f admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:53.893840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761909ed-ef67-4a59-9a1c-5b9e1b9f29fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:53.894733 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761909ed-ef67-4a59-9a1c-5b9e1b9f29fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:53.895081 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761909ed-ef67-4a59-9a1c-5b9e1b9f29fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:53.895361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761909ed-ef67-4a59-9a1c-5b9e1b9f29fe admin admin] PATH_INFO: `/v3/projects/67b06f7f908c4ae19e6955f4c50e7ceb/users/5c5293e7fd8b49638b683090ff024a3f/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:53.897465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761909ed-ef67-4a59-9a1c-5b9e1b9f29fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b06f7f908c4ae19e6955f4c50e7ceb, user_id=5c5293e7fd8b49638b683090ff024a3f, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:53.900175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761909ed-ef67-4a59-9a1c-5b9e1b9f29fe admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:53.911233 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 961/1918] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:24:53 2022] PUT /identity/v3/projects/67b06f7f908c4ae19e6955f4c50e7ceb/users/5c5293e7fd8b49638b683090ff024a3f/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:24:53.921382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b55204-1545-472e-9f71-a5eb48f5f5ac None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:53.921682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b55204-1545-472e-9f71-a5eb48f5f5ac None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:53.921976 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b55204-1545-472e-9f71-a5eb48f5f5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:53.949059 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-b4b55204-1545-472e-9f71-a5eb48f5f5ac None None] MFA Rules not processed for user `5c5293e7fd8b49638b683090ff024a3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:24:53.950274 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-b4b55204-1545-472e-9f71-a5eb48f5f5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:54.019535 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 958/1919] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:24:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:24:54.032628 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761909ed-ef67-4a59-9a1c-5b9e1b9f29fe admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:54.034300 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-761909ed-ef67-4a59-9a1c-5b9e1b9f29fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:54.090535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761909ed-ef67-4a59-9a1c-5b9e1b9f29fe admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:54.094706 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97c8eb-2183-4864-a856-a09c1b355f37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:54.095380 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97c8eb-2183-4864-a856-a09c1b355f37 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:54.095667 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97c8eb-2183-4864-a856-a09c1b355f37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:54.095986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97c8eb-2183-4864-a856-a09c1b355f37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:54.150724 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97c8eb-2183-4864-a856-a09c1b355f37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:54.152147 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97c8eb-2183-4864-a856-a09c1b355f37 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:54.161832 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 962/1920] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:24:54.355431 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:55.121186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:55.469488 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:55.486573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:55.652259 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:55.686606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:55.744510 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:55.995216 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:56.143850 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:56.281611 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:56.555105 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:56.584093 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:56.771735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:56.955784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:57.485815 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b55204-1545-472e-9f71-a5eb48f5f5ac None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:57.485815 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:57.488885 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b55204-1545-472e-9f71-a5eb48f5f5ac None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:57.499769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d29c0f-df5c-4c47-a474-31e5131fed75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:57.501020 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d29c0f-df5c-4c47-a474-31e5131fed75 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:57.501698 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d29c0f-df5c-4c47-a474-31e5131fed75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:57.502005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d29c0f-df5c-4c47-a474-31e5131fed75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:57.567215 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d29c0f-df5c-4c47-a474-31e5131fed75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:57.572463 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d29c0f-df5c-4c47-a474-31e5131fed75 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:57.580331 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 959/1921] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:57.876115 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:58.018882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:58.572865 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:58.778105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:59.366647 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:24:59.450969 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97c8eb-2183-4864-a856-a09c1b355f37 service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:24:59.452564 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-be97c8eb-2183-4864-a856-a09c1b355f37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:24:59.464828 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97c8eb-2183-4864-a856-a09c1b355f37 service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:24:59.468816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e1fd5-0e07-4881-a074-8fbd44f94b87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:24:59.470099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e1fd5-0e07-4881-a074-8fbd44f94b87 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:24:59.470394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e1fd5-0e07-4881-a074-8fbd44f94b87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:24:59.470681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e1fd5-0e07-4881-a074-8fbd44f94b87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:24:59.481561 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e1fd5-0e07-4881-a074-8fbd44f94b87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:24:59.485083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e1fd5-0e07-4881-a074-8fbd44f94b87 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:24:59.495576 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 963/1922] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:24:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:24:59.603403 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:01.973292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d29c0f-df5c-4c47-a474-31e5131fed75 service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:01.974726 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a4d29c0f-df5c-4c47-a474-31e5131fed75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:02.046723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d29c0f-df5c-4c47-a474-31e5131fed75 service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:02.051295 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99fbf33-49d1-4874-82c7-75721370a36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:02.052160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fbf33-49d1-4874-82c7-75721370a36d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:02.052442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fbf33-49d1-4874-82c7-75721370a36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:02.052718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fbf33-49d1-4874-82c7-75721370a36d admin admin] PATH_INFO: `/v3/users/c85838cd819d41da8978dfeebe370a04` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:02.053903 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99fbf33-49d1-4874-82c7-75721370a36d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c85838cd819d41da8978dfeebe370a04)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:02.055969 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99fbf33-49d1-4874-82c7-75721370a36d admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:02.286309 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-c99fbf33-49d1-4874-82c7-75721370a36d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c85838cd819d41da8978dfeebe370a04'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:02.293673 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-c99fbf33-49d1-4874-82c7-75721370a36d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c85838cd819d41da8978dfeebe370a04'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:02.294371 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-c99fbf33-49d1-4874-82c7-75721370a36d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c85838cd819d41da8978dfeebe370a04'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:02.295021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-c99fbf33-49d1-4874-82c7-75721370a36d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c85838cd819d41da8978dfeebe370a04'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:02.307344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-c99fbf33-49d1-4874-82c7-75721370a36d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c85838cd819d41da8978dfeebe370a04'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:02.401401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 960/1923] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:25:01 2022] DELETE /identity/v3/users/c85838cd819d41da8978dfeebe370a04 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:02.413695 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abcc24d-08a9-4c7f-8331-195b74be6c18 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:02.414180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abcc24d-08a9-4c7f-8331-195b74be6c18 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:02.414346 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abcc24d-08a9-4c7f-8331-195b74be6c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:02.433119 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-7abcc24d-08a9-4c7f-8331-195b74be6c18 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:02.434325 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7abcc24d-08a9-4c7f-8331-195b74be6c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:02.440241 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:02.498747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 964/1924] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:02.508960 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99fbf33-49d1-4874-82c7-75721370a36d admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:02.510369 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-c99fbf33-49d1-4874-82c7-75721370a36d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:02.542876 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:02.569090 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99fbf33-49d1-4874-82c7-75721370a36d admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:02.572751 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340c919-f921-4085-818d-e0dcd159a5e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:02.573400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340c919-f921-4085-818d-e0dcd159a5e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:02.573713 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340c919-f921-4085-818d-e0dcd159a5e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:02.573991 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340c919-f921-4085-818d-e0dcd159a5e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:02.583891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340c919-f921-4085-818d-e0dcd159a5e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:02.585367 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340c919-f921-4085-818d-e0dcd159a5e8 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:02.597238 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 961/1925] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:25:03.163668 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abcc24d-08a9-4c7f-8331-195b74be6c18 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:03.165762 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7abcc24d-08a9-4c7f-8331-195b74be6c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:03.178628 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abcc24d-08a9-4c7f-8331-195b74be6c18 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:03.183445 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6c053-7276-4486-9893-cde88934bc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:03.184457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6c053-7276-4486-9893-cde88934bc14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:03.184765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6c053-7276-4486-9893-cde88934bc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:03.185110 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6c053-7276-4486-9893-cde88934bc14 admin admin] PATH_INFO: `/v3/projects/0034c51704ca48ef90af2caa66968099` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:03.186390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6c053-7276-4486-9893-cde88934bc14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0034c51704ca48ef90af2caa66968099)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:03.188842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6c053-7276-4486-9893-cde88934bc14 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:03.290984 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b9d6c053-7276-4486-9893-cde88934bc14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0034c51704ca48ef90af2caa66968099', 'request_id': 'req-b9d6c053-7276-4486-9893-cde88934bc14'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:03.300729 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b9d6c053-7276-4486-9893-cde88934bc14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0034c51704ca48ef90af2caa66968099', 'request_id': 'req-b9d6c053-7276-4486-9893-cde88934bc14'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:03.311829 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b9d6c053-7276-4486-9893-cde88934bc14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0034c51704ca48ef90af2caa66968099', 'request_id': 'req-b9d6c053-7276-4486-9893-cde88934bc14'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:03.469571 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b9d6c053-7276-4486-9893-cde88934bc14 admin admin] The token cache is being invalidate because project 0034c51704ca48ef90af2caa66968099 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:25:03.469571 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b9d6c053-7276-4486-9893-cde88934bc14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:03.469571 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b9d6c053-7276-4486-9893-cde88934bc14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:03.469571 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 965/1926] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:25:03 2022] DELETE /identity/v3/projects/0034c51704ca48ef90af2caa66968099 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:03.472232 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340c919-f921-4085-818d-e0dcd159a5e8 service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:03.482203 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340c919-f921-4085-818d-e0dcd159a5e8 service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:03.543779 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a34c26-fd87-4418-86de-a61417abc02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:03.544757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a34c26-fd87-4418-86de-a61417abc02b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:03.545117 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a34c26-fd87-4418-86de-a61417abc02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:03.545400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a34c26-fd87-4418-86de-a61417abc02b admin admin] PATH_INFO: `/v3/users/0bc0adf48e9a47d0a7b5c137a34a5916` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:03.546685 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a34c26-fd87-4418-86de-a61417abc02b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bc0adf48e9a47d0a7b5c137a34a5916)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:03.549483 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a34c26-fd87-4418-86de-a61417abc02b admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:03.826735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d4a34c26-fd87-4418-86de-a61417abc02b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bc0adf48e9a47d0a7b5c137a34a5916'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:03.833646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d4a34c26-fd87-4418-86de-a61417abc02b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bc0adf48e9a47d0a7b5c137a34a5916'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:03.834358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d4a34c26-fd87-4418-86de-a61417abc02b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bc0adf48e9a47d0a7b5c137a34a5916'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:03.835021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d4a34c26-fd87-4418-86de-a61417abc02b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bc0adf48e9a47d0a7b5c137a34a5916'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:03.849145 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d4a34c26-fd87-4418-86de-a61417abc02b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bc0adf48e9a47d0a7b5c137a34a5916'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:03.972291 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 962/1927] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:25:03 2022] DELETE /identity/v3/users/0bc0adf48e9a47d0a7b5c137a34a5916 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:04.445126 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6c053-7276-4486-9893-cde88934bc14 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:04.446806 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b9d6c053-7276-4486-9893-cde88934bc14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:04.475467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6c053-7276-4486-9893-cde88934bc14 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:04.549617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eaba17-74a7-433a-9bb5-02bb9101a233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:04.550616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eaba17-74a7-433a-9bb5-02bb9101a233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:04.550906 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eaba17-74a7-433a-9bb5-02bb9101a233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:04.551216 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eaba17-74a7-433a-9bb5-02bb9101a233 admin admin] PATH_INFO: `/v3/projects/b8242a3bcd4444bbb162274ae10bd3b5` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:04.552409 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eaba17-74a7-433a-9bb5-02bb9101a233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8242a3bcd4444bbb162274ae10bd3b5)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:04.554311 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eaba17-74a7-433a-9bb5-02bb9101a233 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:04.697499 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:04.706081 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-27eaba17-74a7-433a-9bb5-02bb9101a233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8242a3bcd4444bbb162274ae10bd3b5', 'request_id': 'req-27eaba17-74a7-433a-9bb5-02bb9101a233'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:04.712267 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-27eaba17-74a7-433a-9bb5-02bb9101a233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8242a3bcd4444bbb162274ae10bd3b5', 'request_id': 'req-27eaba17-74a7-433a-9bb5-02bb9101a233'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:04.718099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-27eaba17-74a7-433a-9bb5-02bb9101a233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8242a3bcd4444bbb162274ae10bd3b5', 'request_id': 'req-27eaba17-74a7-433a-9bb5-02bb9101a233'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:04.863426 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-27eaba17-74a7-433a-9bb5-02bb9101a233 admin admin] The token cache is being invalidate because project b8242a3bcd4444bbb162274ae10bd3b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:25:04.863713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-27eaba17-74a7-433a-9bb5-02bb9101a233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:04.864405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-27eaba17-74a7-433a-9bb5-02bb9101a233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:04.865992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 966/1928] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:25:04 2022] DELETE /identity/v3/projects/b8242a3bcd4444bbb162274ae10bd3b5 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:04.872084 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0f790-759d-4cf8-939b-899b94da480c None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:04.872871 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0f790-759d-4cf8-939b-899b94da480c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:04.872871 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0f790-759d-4cf8-939b-899b94da480c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:04.892376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-b7d0f790-759d-4cf8-939b-899b94da480c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:04.893599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-b7d0f790-759d-4cf8-939b-899b94da480c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:04.950863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:04.976156 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 963/1929] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:25:04.984341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4d32e-d630-4619-9ae3-68323bf1eb59 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:04.985019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4d32e-d630-4619-9ae3-68323bf1eb59 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:04.985304 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4d32e-d630-4619-9ae3-68323bf1eb59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:04.998419 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0f790-759d-4cf8-939b-899b94da480c None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:05.000330 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-b7d0f790-759d-4cf8-939b-899b94da480c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:05.012980 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-f0a4d32e-d630-4619-9ae3-68323bf1eb59 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:05.014421 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-f0a4d32e-d630-4619-9ae3-68323bf1eb59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:05.016992 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0f790-759d-4cf8-939b-899b94da480c None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:05.022054 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d1f34-65d8-4db7-abe1-e621d1e39a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:05.023530 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d1f34-65d8-4db7-abe1-e621d1e39a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:05.024003 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d1f34-65d8-4db7-abe1-e621d1e39a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:05.024289 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d1f34-65d8-4db7-abe1-e621d1e39a87 admin admin] PATH_INFO: `/v3/users/e536b7083a964148a5bdffe90aada4be` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:05.025545 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d1f34-65d8-4db7-abe1-e621d1e39a87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e536b7083a964148a5bdffe90aada4be)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:05.027586 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d1f34-65d8-4db7-abe1-e621d1e39a87 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:05.030295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 967/1930] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:05.035326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4d32e-d630-4619-9ae3-68323bf1eb59 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:05.047745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4d32e-d630-4619-9ae3-68323bf1eb59 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:05.055227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89475469-b71f-48a5-a86b-42c958939500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:05.056039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89475469-b71f-48a5-a86b-42c958939500 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:05.056827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89475469-b71f-48a5-a86b-42c958939500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:05.057655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89475469-b71f-48a5-a86b-42c958939500 admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:05.059384 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89475469-b71f-48a5-a86b-42c958939500 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:25:05.066708 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89475469-b71f-48a5-a86b-42c958939500 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:05.066708 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89475469-b71f-48a5-a86b-42c958939500 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:05.075309 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 968/1931] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:25:05.080653 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89475469-b71f-48a5-a86b-42c958939500 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:05.091324 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89475469-b71f-48a5-a86b-42c958939500 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:05.096560 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e11f97-b772-4d33-8fd8-562a3a64ab25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:05.097407 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e11f97-b772-4d33-8fd8-562a3a64ab25 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:05.097698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e11f97-b772-4d33-8fd8-562a3a64ab25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:05.097980 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e11f97-b772-4d33-8fd8-562a3a64ab25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:05.098678 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e11f97-b772-4d33-8fd8-562a3a64ab25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:05.100637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e11f97-b772-4d33-8fd8-562a3a64ab25 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:05.236441 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 969/1932] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:25:05 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:25:05.240355 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e11f97-b772-4d33-8fd8-562a3a64ab25 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:05.271362 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f70d1f34-65d8-4db7-abe1-e621d1e39a87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e536b7083a964148a5bdffe90aada4be'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:05.278277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f70d1f34-65d8-4db7-abe1-e621d1e39a87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e536b7083a964148a5bdffe90aada4be'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:05.278974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f70d1f34-65d8-4db7-abe1-e621d1e39a87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e536b7083a964148a5bdffe90aada4be'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:05.279594 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f70d1f34-65d8-4db7-abe1-e621d1e39a87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e536b7083a964148a5bdffe90aada4be'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:05.291776 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f70d1f34-65d8-4db7-abe1-e621d1e39a87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e536b7083a964148a5bdffe90aada4be'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:05.306394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e11f97-b772-4d33-8fd8-562a3a64ab25 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:05.311557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd98cac-c3b5-40e7-be1c-332766aa60fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:05.312919 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd98cac-c3b5-40e7-be1c-332766aa60fa admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:05.313262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd98cac-c3b5-40e7-be1c-332766aa60fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:05.313619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd98cac-c3b5-40e7-be1c-332766aa60fa admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:05.314308 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd98cac-c3b5-40e7-be1c-332766aa60fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:05.316262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd98cac-c3b5-40e7-be1c-332766aa60fa admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:05.396538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 964/1933] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:25:04 2022] DELETE /identity/v3/users/e536b7083a964148a5bdffe90aada4be => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:25:05.401141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f7194-9774-4247-8a61-564255618b31 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:05.401467 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f7194-9774-4247-8a61-564255618b31 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:05.401759 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f7194-9774-4247-8a61-564255618b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:05.423686 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 970/1934] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:25:05 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:25:05.434631 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-a78f7194-9774-4247-8a61-564255618b31 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:05.434631 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a78f7194-9774-4247-8a61-564255618b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:05.439170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd98cac-c3b5-40e7-be1c-332766aa60fa admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:05.440922 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-5fd98cac-c3b5-40e7-be1c-332766aa60fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:05.454114 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd98cac-c3b5-40e7-be1c-332766aa60fa admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:05.508734 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 965/1935] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:25:05.518952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f7194-9774-4247-8a61-564255618b31 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:05.520621 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a78f7194-9774-4247-8a61-564255618b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:05.523534 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c1729-17f3-4a00-9f1a-333cba9208f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:05.524232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c1729-17f3-4a00-9f1a-333cba9208f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:05.524528 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c1729-17f3-4a00-9f1a-333cba9208f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:05.525008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c1729-17f3-4a00-9f1a-333cba9208f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:05.525554 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c1729-17f3-4a00-9f1a-333cba9208f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:05.527245 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c1729-17f3-4a00-9f1a-333cba9208f2 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:05.538935 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 971/1936] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:25:05.542864 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c1729-17f3-4a00-9f1a-333cba9208f2 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:05.553174 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c1729-17f3-4a00-9f1a-333cba9208f2 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:05.557845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3079602-d69c-419e-9d5a-d27240004032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:05.558496 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3079602-d69c-419e-9d5a-d27240004032 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:05.558786 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3079602-d69c-419e-9d5a-d27240004032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:05.559139 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3079602-d69c-419e-9d5a-d27240004032 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:05.560081 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3079602-d69c-419e-9d5a-d27240004032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:05.561687 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3079602-d69c-419e-9d5a-d27240004032 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:05.564419 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:05.574004 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 972/1937] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:25:05.578561 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3079602-d69c-419e-9d5a-d27240004032 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:05.586547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f7194-9774-4247-8a61-564255618b31 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:05.589266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3079602-d69c-419e-9d5a-d27240004032 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:05.590618 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df03bf7-20e2-4f7e-ac0c-f100687da787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:05.600495 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df03bf7-20e2-4f7e-ac0c-f100687da787 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:05.600495 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df03bf7-20e2-4f7e-ac0c-f100687da787 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:05.600495 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df03bf7-20e2-4f7e-ac0c-f100687da787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:05.600780 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0f82b-f20c-47cf-8ab7-072ef03c6334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:05.600780 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0f82b-f20c-47cf-8ab7-072ef03c6334 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:05.600780 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0f82b-f20c-47cf-8ab7-072ef03c6334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:05.600780 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0f82b-f20c-47cf-8ab7-072ef03c6334 admin admin] PATH_INFO: `/v3/projects/fde682346eb14236967f36a6aea93686/users/30cfb7257a0644a3a2c75bd2a0c43559/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:05.610460 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df03bf7-20e2-4f7e-ac0c-f100687da787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:05.611961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df03bf7-20e2-4f7e-ac0c-f100687da787 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:05.616457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0f82b-f20c-47cf-8ab7-072ef03c6334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fde682346eb14236967f36a6aea93686, user_id=30cfb7257a0644a3a2c75bd2a0c43559, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:05.619133 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0f82b-f20c-47cf-8ab7-072ef03c6334 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:05.622285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 966/1938] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:25:05.755892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 973/1939] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:25:05 2022] PUT /identity/v3/projects/fde682346eb14236967f36a6aea93686/users/30cfb7257a0644a3a2c75bd2a0c43559/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:25:05.766680 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f850e45-c1ef-44dc-8af5-aebb4f4c694a None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:05.767108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f850e45-c1ef-44dc-8af5-aebb4f4c694a None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:05.767559 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f850e45-c1ef-44dc-8af5-aebb4f4c694a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:05.775146 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:05.797083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-3f850e45-c1ef-44dc-8af5-aebb4f4c694a None None] MFA Rules not processed for user `30cfb7257a0644a3a2c75bd2a0c43559`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:05.801470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-3f850e45-c1ef-44dc-8af5-aebb4f4c694a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:05.867087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 967/1940] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:05.871658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0f82b-f20c-47cf-8ab7-072ef03c6334 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:05.939158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0f82b-f20c-47cf-8ab7-072ef03c6334 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:05.949590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab5355-5584-4fde-9bce-02e11f1486a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:05.950348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab5355-5584-4fde-9bce-02e11f1486a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:05.950706 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab5355-5584-4fde-9bce-02e11f1486a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:05.951408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab5355-5584-4fde-9bce-02e11f1486a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:05.952186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab5355-5584-4fde-9bce-02e11f1486a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:05.954751 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab5355-5584-4fde-9bce-02e11f1486a0 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:06.069150 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 974/1941] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:25:05 2022] POST /identity/v3/projects => generated 373 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:25:06.073656 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f850e45-c1ef-44dc-8af5-aebb4f4c694a None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:06.084664 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f850e45-c1ef-44dc-8af5-aebb4f4c694a None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:06.143754 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f2413-1f6c-4236-ab7f-e0a36d722eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:06.144446 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f2413-1f6c-4236-ab7f-e0a36d722eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:06.144738 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f2413-1f6c-4236-ab7f-e0a36d722eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:06.145020 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f2413-1f6c-4236-ab7f-e0a36d722eb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:06.145734 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f2413-1f6c-4236-ab7f-e0a36d722eb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:06.147813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f2413-1f6c-4236-ab7f-e0a36d722eb7 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:06.167895 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab5355-5584-4fde-9bce-02e11f1486a0 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:06.169742 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e8ab5355-5584-4fde-9bce-02e11f1486a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:06.183290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab5355-5584-4fde-9bce-02e11f1486a0 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:06.190142 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ed6c6-2f0a-4671-93e3-3ace583f1634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:06.191135 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ed6c6-2f0a-4671-93e3-3ace583f1634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:06.191427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ed6c6-2f0a-4671-93e3-3ace583f1634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:06.191732 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ed6c6-2f0a-4671-93e3-3ace583f1634 admin admin] PATH_INFO: `/v3/projects/41e68f66f1674c44a68c89df35df0609` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:06.192897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ed6c6-2f0a-4671-93e3-3ace583f1634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41e68f66f1674c44a68c89df35df0609)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:06.194858 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ed6c6-2f0a-4671-93e3-3ace583f1634 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:06.288650 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 968/1942] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:25:06 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:25:06.292332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f2413-1f6c-4236-ab7f-e0a36d722eb7 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:06.304507 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f2413-1f6c-4236-ab7f-e0a36d722eb7 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:06.318982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-777ed6c6-2f0a-4671-93e3-3ace583f1634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41e68f66f1674c44a68c89df35df0609', 'request_id': 'req-777ed6c6-2f0a-4671-93e3-3ace583f1634'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:06.325640 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-777ed6c6-2f0a-4671-93e3-3ace583f1634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41e68f66f1674c44a68c89df35df0609', 'request_id': 'req-777ed6c6-2f0a-4671-93e3-3ace583f1634'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:06.331653 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-777ed6c6-2f0a-4671-93e3-3ace583f1634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41e68f66f1674c44a68c89df35df0609', 'request_id': 'req-777ed6c6-2f0a-4671-93e3-3ace583f1634'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:06.361393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e1382-66ca-49b6-b0ee-0ea1eb0e1527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:06.362097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e1382-66ca-49b6-b0ee-0ea1eb0e1527 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:06.362392 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e1382-66ca-49b6-b0ee-0ea1eb0e1527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:06.364958 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e1382-66ca-49b6-b0ee-0ea1eb0e1527 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:06.366069 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e1382-66ca-49b6-b0ee-0ea1eb0e1527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:06.367813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e1382-66ca-49b6-b0ee-0ea1eb0e1527 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:06.381630 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 969/1943] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:25:06.385646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e1382-66ca-49b6-b0ee-0ea1eb0e1527 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:06.396663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e1382-66ca-49b6-b0ee-0ea1eb0e1527 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:06.401681 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5ff93-7a0a-4061-89b2-7afa7729c736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:06.402622 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5ff93-7a0a-4061-89b2-7afa7729c736 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:06.402905 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5ff93-7a0a-4061-89b2-7afa7729c736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:06.403298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5ff93-7a0a-4061-89b2-7afa7729c736 admin admin] PATH_INFO: `/v3/projects/2d694507867c4c46940a94b1262c2aaf/users/036b13ae809d404c88020fd383cfde3d/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:06.423622 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5ff93-7a0a-4061-89b2-7afa7729c736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d694507867c4c46940a94b1262c2aaf, user_id=036b13ae809d404c88020fd383cfde3d, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:06.426293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5ff93-7a0a-4061-89b2-7afa7729c736 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:06.440864 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-777ed6c6-2f0a-4671-93e3-3ace583f1634 admin admin] The token cache is being invalidate because project 41e68f66f1674c44a68c89df35df0609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:25:06.441165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-777ed6c6-2f0a-4671-93e3-3ace583f1634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:06.442137 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-777ed6c6-2f0a-4671-93e3-3ace583f1634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:06.444157 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 975/1944] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:25:06 2022] DELETE /identity/v3/projects/41e68f66f1674c44a68c89df35df0609 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:25:06.448117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ed6c6-2f0a-4671-93e3-3ace583f1634 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:06.450434 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-777ed6c6-2f0a-4671-93e3-3ace583f1634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:06.463706 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ed6c6-2f0a-4671-93e3-3ace583f1634 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:06.468551 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee85768-5f21-467f-b809-dd172308a4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:06.469500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee85768-5f21-467f-b809-dd172308a4a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:06.469822 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee85768-5f21-467f-b809-dd172308a4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:06.470050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee85768-5f21-467f-b809-dd172308a4a7 admin admin] PATH_INFO: `/v3/users/78c278badeb248c394f5c1ebc9444fee` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:06.471314 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee85768-5f21-467f-b809-dd172308a4a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78c278badeb248c394f5c1ebc9444fee)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:06.473334 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee85768-5f21-467f-b809-dd172308a4a7 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:06.544246 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 970/1945] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:25:06 2022] PUT /identity/v3/projects/2d694507867c4c46940a94b1262c2aaf/users/036b13ae809d404c88020fd383cfde3d/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:25:06.548540 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5ff93-7a0a-4061-89b2-7afa7729c736 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:06.550114 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-e9d5ff93-7a0a-4061-89b2-7afa7729c736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:06.575960 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:06.628752 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5ff93-7a0a-4061-89b2-7afa7729c736 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:06.633411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45750f5-6a73-4cdd-9217-bbd47c0fdaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:06.634151 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45750f5-6a73-4cdd-9217-bbd47c0fdaf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:06.634435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45750f5-6a73-4cdd-9217-bbd47c0fdaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:06.634715 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45750f5-6a73-4cdd-9217-bbd47c0fdaf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:06.635481 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45750f5-6a73-4cdd-9217-bbd47c0fdaf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:06.637126 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45750f5-6a73-4cdd-9217-bbd47c0fdaf9 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:06.647023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 971/1946] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:25:06.651069 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45750f5-6a73-4cdd-9217-bbd47c0fdaf9 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:06.661078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45750f5-6a73-4cdd-9217-bbd47c0fdaf9 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:06.666144 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc0379-2b8c-4759-9d86-5ac5e1d49926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:06.667070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc0379-2b8c-4759-9d86-5ac5e1d49926 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:06.667364 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc0379-2b8c-4759-9d86-5ac5e1d49926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:06.667933 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc0379-2b8c-4759-9d86-5ac5e1d49926 admin admin] PATH_INFO: `/v3/projects/2d694507867c4c46940a94b1262c2aaf/users/036b13ae809d404c88020fd383cfde3d/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:06.670194 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc0379-2b8c-4759-9d86-5ac5e1d49926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d694507867c4c46940a94b1262c2aaf, user_id=036b13ae809d404c88020fd383cfde3d, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:06.673123 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc0379-2b8c-4759-9d86-5ac5e1d49926 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:06.680257 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=85,keystone:INSERT=3,keystone:DELETE=12 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:06.684810 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 972/1947] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:25:06 2022] PUT /identity/v3/projects/2d694507867c4c46940a94b1262c2aaf/users/036b13ae809d404c88020fd383cfde3d/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:25:06.690124 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-eee85768-5f21-467f-b809-dd172308a4a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78c278badeb248c394f5c1ebc9444fee'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:06.695367 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff53b86-29fe-451a-b7b8-070c49cb3441 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:06.695663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff53b86-29fe-451a-b7b8-070c49cb3441 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:06.695949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff53b86-29fe-451a-b7b8-070c49cb3441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:06.699879 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-eee85768-5f21-467f-b809-dd172308a4a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78c278badeb248c394f5c1ebc9444fee'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:06.700578 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-eee85768-5f21-467f-b809-dd172308a4a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78c278badeb248c394f5c1ebc9444fee'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:06.701199 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-eee85768-5f21-467f-b809-dd172308a4a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78c278badeb248c394f5c1ebc9444fee'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:06.717304 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-eee85768-5f21-467f-b809-dd172308a4a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78c278badeb248c394f5c1ebc9444fee'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:06.725185 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-eff53b86-29fe-451a-b7b8-070c49cb3441 None None] MFA Rules not processed for user `036b13ae809d404c88020fd383cfde3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:06.726409 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-eff53b86-29fe-451a-b7b8-070c49cb3441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:06.798020 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 973/1948] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:06.801405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 976/1949] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:25:06 2022] DELETE /identity/v3/users/78c278badeb248c394f5c1ebc9444fee => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:06.811057 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff53b86-29fe-451a-b7b8-070c49cb3441 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:06.812508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-eff53b86-29fe-451a-b7b8-070c49cb3441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:06.884625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff53b86-29fe-451a-b7b8-070c49cb3441 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:06.901932 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:06.936687 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4c9b2-5a1d-463c-a801-a53be4d1a3bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:06.937877 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4c9b2-5a1d-463c-a801-a53be4d1a3bb service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:06.938180 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4c9b2-5a1d-463c-a801-a53be4d1a3bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:06.938462 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4c9b2-5a1d-463c-a801-a53be4d1a3bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:06.940410 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-2aa4c9b2-5a1d-463c-a801-a53be4d1a3bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:07.001797 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa4c9b2-5a1d-463c-a801-a53be4d1a3bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:07.003632 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa4c9b2-5a1d-463c-a801-a53be4d1a3bb service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:07.023101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 974/1950] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:25:07.339532 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee85768-5f21-467f-b809-dd172308a4a7 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:07.341590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-eee85768-5f21-467f-b809-dd172308a4a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:07.416273 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee85768-5f21-467f-b809-dd172308a4a7 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:07.421004 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f58098-1555-41d7-85f6-d52bbd6c92fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:07.422492 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f58098-1555-41d7-85f6-d52bbd6c92fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:07.422790 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f58098-1555-41d7-85f6-d52bbd6c92fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:07.423103 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f58098-1555-41d7-85f6-d52bbd6c92fc admin admin] PATH_INFO: `/v3/projects/c4b794e3f4324a22be66224eafad3be6` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:07.424312 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f58098-1555-41d7-85f6-d52bbd6c92fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4b794e3f4324a22be66224eafad3be6)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:07.426364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f58098-1555-41d7-85f6-d52bbd6c92fc admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:07.555813 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e7f58098-1555-41d7-85f6-d52bbd6c92fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4b794e3f4324a22be66224eafad3be6', 'request_id': 'req-e7f58098-1555-41d7-85f6-d52bbd6c92fc'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:07.563374 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e7f58098-1555-41d7-85f6-d52bbd6c92fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4b794e3f4324a22be66224eafad3be6', 'request_id': 'req-e7f58098-1555-41d7-85f6-d52bbd6c92fc'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:07.570837 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e7f58098-1555-41d7-85f6-d52bbd6c92fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4b794e3f4324a22be66224eafad3be6', 'request_id': 'req-e7f58098-1555-41d7-85f6-d52bbd6c92fc'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:07.731252 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e7f58098-1555-41d7-85f6-d52bbd6c92fc admin admin] The token cache is being invalidate because project c4b794e3f4324a22be66224eafad3be6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:25:07.731252 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e7f58098-1555-41d7-85f6-d52bbd6c92fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:07.731252 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e7f58098-1555-41d7-85f6-d52bbd6c92fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:07.731252 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 977/1951] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:25:07 2022] DELETE /identity/v3/projects/c4b794e3f4324a22be66224eafad3be6 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:07.732471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4c9b2-5a1d-463c-a801-a53be4d1a3bb service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:07.743680 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4c9b2-5a1d-463c-a801-a53be4d1a3bb service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:07.804800 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a44cbb6-024f-4c41-b3fe-b0fdfc8fc82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:07.805689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44cbb6-024f-4c41-b3fe-b0fdfc8fc82b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:07.805982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44cbb6-024f-4c41-b3fe-b0fdfc8fc82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:07.806267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44cbb6-024f-4c41-b3fe-b0fdfc8fc82b admin admin] PATH_INFO: `/v3/users/5c5293e7fd8b49638b683090ff024a3f` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:07.807518 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a44cbb6-024f-4c41-b3fe-b0fdfc8fc82b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c5293e7fd8b49638b683090ff024a3f)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:07.809488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a44cbb6-024f-4c41-b3fe-b0fdfc8fc82b admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:08.068955 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2a44cbb6-024f-4c41-b3fe-b0fdfc8fc82b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c5293e7fd8b49638b683090ff024a3f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:08.077081 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2a44cbb6-024f-4c41-b3fe-b0fdfc8fc82b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c5293e7fd8b49638b683090ff024a3f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:08.078037 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2a44cbb6-024f-4c41-b3fe-b0fdfc8fc82b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c5293e7fd8b49638b683090ff024a3f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:08.079287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2a44cbb6-024f-4c41-b3fe-b0fdfc8fc82b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c5293e7fd8b49638b683090ff024a3f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:08.092958 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2a44cbb6-024f-4c41-b3fe-b0fdfc8fc82b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c5293e7fd8b49638b683090ff024a3f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:08.095814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:08.206717 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 975/1952] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:25:07 2022] DELETE /identity/v3/users/5c5293e7fd8b49638b683090ff024a3f => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:08.676754 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f58098-1555-41d7-85f6-d52bbd6c92fc admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:08.678289 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e7f58098-1555-41d7-85f6-d52bbd6c92fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:08.692068 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f58098-1555-41d7-85f6-d52bbd6c92fc admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:08.733610 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:08.748447 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1be40f-4cb7-4fb4-a7f1-4b0f529aaa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:08.749310 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1be40f-4cb7-4fb4-a7f1-4b0f529aaa44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:08.749603 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1be40f-4cb7-4fb4-a7f1-4b0f529aaa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:08.749899 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1be40f-4cb7-4fb4-a7f1-4b0f529aaa44 admin admin] PATH_INFO: `/v3/projects/67b06f7f908c4ae19e6955f4c50e7ceb` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:08.751664 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1be40f-4cb7-4fb4-a7f1-4b0f529aaa44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67b06f7f908c4ae19e6955f4c50e7ceb)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:08.753686 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1be40f-4cb7-4fb4-a7f1-4b0f529aaa44 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:08.853707 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-0d1be40f-4cb7-4fb4-a7f1-4b0f529aaa44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67b06f7f908c4ae19e6955f4c50e7ceb', 'request_id': 'req-0d1be40f-4cb7-4fb4-a7f1-4b0f529aaa44'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:08.860801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-0d1be40f-4cb7-4fb4-a7f1-4b0f529aaa44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67b06f7f908c4ae19e6955f4c50e7ceb', 'request_id': 'req-0d1be40f-4cb7-4fb4-a7f1-4b0f529aaa44'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:08.866949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-0d1be40f-4cb7-4fb4-a7f1-4b0f529aaa44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67b06f7f908c4ae19e6955f4c50e7ceb', 'request_id': 'req-0d1be40f-4cb7-4fb4-a7f1-4b0f529aaa44'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:08.965533 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-0d1be40f-4cb7-4fb4-a7f1-4b0f529aaa44 admin admin] The token cache is being invalidate because project 67b06f7f908c4ae19e6955f4c50e7ceb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:25:08.965830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-0d1be40f-4cb7-4fb4-a7f1-4b0f529aaa44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:08.966526 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-0d1be40f-4cb7-4fb4-a7f1-4b0f529aaa44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:08.968106 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 978/1953] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:25:08 2022] DELETE /identity/v3/projects/67b06f7f908c4ae19e6955f4c50e7ceb => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:08.974875 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41bcfd-9cf3-4f94-a8f9-8ec47f95c910 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:08.975357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41bcfd-9cf3-4f94-a8f9-8ec47f95c910 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:08.975601 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41bcfd-9cf3-4f94-a8f9-8ec47f95c910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:08.993790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-5e41bcfd-9cf3-4f94-a8f9-8ec47f95c910 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:08.995004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5e41bcfd-9cf3-4f94-a8f9-8ec47f95c910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:09.055829 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 976/1954] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:25:09.063650 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2aa315-2495-45a8-a1eb-644f3764617e None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:09.063942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2aa315-2495-45a8-a1eb-644f3764617e None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:09.064221 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2aa315-2495-45a8-a1eb-644f3764617e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:09.082711 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-ee2aa315-2495-45a8-a1eb-644f3764617e None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:09.084000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-ee2aa315-2495-45a8-a1eb-644f3764617e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:09.091895 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 979/1955] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:09.096557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41bcfd-9cf3-4f94-a8f9-8ec47f95c910 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:09.106138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41bcfd-9cf3-4f94-a8f9-8ec47f95c910 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:09.111092 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078e872-502b-4eca-9ddd-5070c09b7336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:09.111744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078e872-502b-4eca-9ddd-5070c09b7336 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:09.112033 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078e872-502b-4eca-9ddd-5070c09b7336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:09.121884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078e872-502b-4eca-9ddd-5070c09b7336 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:09.121884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078e872-502b-4eca-9ddd-5070c09b7336 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:25:09.121884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078e872-502b-4eca-9ddd-5070c09b7336 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:09.121884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078e872-502b-4eca-9ddd-5070c09b7336 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:09.127805 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 977/1956] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:25:09.131754 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2aa315-2495-45a8-a1eb-644f3764617e None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:09.141484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2aa315-2495-45a8-a1eb-644f3764617e None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:09.146190 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163592e9-53ec-45c0-b28b-19192edb7714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:09.146845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163592e9-53ec-45c0-b28b-19192edb7714 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:09.147235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163592e9-53ec-45c0-b28b-19192edb7714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:09.147517 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163592e9-53ec-45c0-b28b-19192edb7714 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:09.148220 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163592e9-53ec-45c0-b28b-19192edb7714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:09.150212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163592e9-53ec-45c0-b28b-19192edb7714 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:09.254341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 980/1957] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:25:09 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:25:09.258529 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078e872-502b-4eca-9ddd-5070c09b7336 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:09.268595 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078e872-502b-4eca-9ddd-5070c09b7336 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:09.334808 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aab3d4-ec1c-41f7-b7e9-4f5e464dc537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:09.335512 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aab3d4-ec1c-41f7-b7e9-4f5e464dc537 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:09.335816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aab3d4-ec1c-41f7-b7e9-4f5e464dc537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:09.336445 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aab3d4-ec1c-41f7-b7e9-4f5e464dc537 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:09.337168 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aab3d4-ec1c-41f7-b7e9-4f5e464dc537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:09.339139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aab3d4-ec1c-41f7-b7e9-4f5e464dc537 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:09.463517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 978/1958] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:25:09 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:25:09.467663 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163592e9-53ec-45c0-b28b-19192edb7714 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:09.478713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163592e9-53ec-45c0-b28b-19192edb7714 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:09.483501 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60001e7-d1cc-4036-8b39-60d40be40a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:09.484679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60001e7-d1cc-4036-8b39-60d40be40a13 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:09.485017 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60001e7-d1cc-4036-8b39-60d40be40a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:09.485293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60001e7-d1cc-4036-8b39-60d40be40a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:09.486026 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60001e7-d1cc-4036-8b39-60d40be40a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:09.487647 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60001e7-d1cc-4036-8b39-60d40be40a13 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:09.498008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 981/1959] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:25:09.502460 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aab3d4-ec1c-41f7-b7e9-4f5e464dc537 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:09.512514 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aab3d4-ec1c-41f7-b7e9-4f5e464dc537 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:09.517208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba1ab7-74e9-41a4-8eb1-991222e295d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:09.517861 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba1ab7-74e9-41a4-8eb1-991222e295d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:09.518150 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba1ab7-74e9-41a4-8eb1-991222e295d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:09.518427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba1ab7-74e9-41a4-8eb1-991222e295d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:09.519185 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba1ab7-74e9-41a4-8eb1-991222e295d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:09.520786 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba1ab7-74e9-41a4-8eb1-991222e295d1 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:09.530513 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 979/1960] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:25:09.540703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60001e7-d1cc-4036-8b39-60d40be40a13 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:09.551149 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60001e7-d1cc-4036-8b39-60d40be40a13 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:09.555717 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ccbf7-6ea7-4c4e-8732-a5ea2a15e003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:09.556843 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ccbf7-6ea7-4c4e-8732-a5ea2a15e003 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:09.557180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ccbf7-6ea7-4c4e-8732-a5ea2a15e003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:09.557488 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ccbf7-6ea7-4c4e-8732-a5ea2a15e003 admin admin] PATH_INFO: `/v3/projects/56c1fd17565b4ec283f7a86ae7bd71a4/users/0051a1bd520e47229d232c7988b591bb/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:09.572392 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ccbf7-6ea7-4c4e-8732-a5ea2a15e003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56c1fd17565b4ec283f7a86ae7bd71a4, user_id=0051a1bd520e47229d232c7988b591bb, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:09.575065 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ccbf7-6ea7-4c4e-8732-a5ea2a15e003 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:09.732574 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 982/1961] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:25:09 2022] PUT /identity/v3/projects/56c1fd17565b4ec283f7a86ae7bd71a4/users/0051a1bd520e47229d232c7988b591bb/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:09.743518 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3155d3-9b1d-46a4-ba53-9a75e0b18d2c None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:09.743820 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3155d3-9b1d-46a4-ba53-9a75e0b18d2c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:09.744105 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3155d3-9b1d-46a4-ba53-9a75e0b18d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:09.769900 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-2a3155d3-9b1d-46a4-ba53-9a75e0b18d2c None None] MFA Rules not processed for user `0051a1bd520e47229d232c7988b591bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:09.772866 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-2a3155d3-9b1d-46a4-ba53-9a75e0b18d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:09.817630 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:09.838339 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 980/1962] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:09.846564 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ccbf7-6ea7-4c4e-8732-a5ea2a15e003 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:09.848292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-722ccbf7-6ea7-4c4e-8732-a5ea2a15e003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:09.907272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ccbf7-6ea7-4c4e-8732-a5ea2a15e003 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:09.910881 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45707df-1560-41b3-998a-0b9bd3791244 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:09.911561 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45707df-1560-41b3-998a-0b9bd3791244 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:09.911930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45707df-1560-41b3-998a-0b9bd3791244 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:09.912217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45707df-1560-41b3-998a-0b9bd3791244 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:09.920766 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45707df-1560-41b3-998a-0b9bd3791244 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:09.923086 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45707df-1560-41b3-998a-0b9bd3791244 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:09.932065 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 983/1963] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:25:13.965455 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3155d3-9b1d-46a4-ba53-9a75e0b18d2c None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:13.966881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-2a3155d3-9b1d-46a4-ba53-9a75e0b18d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:14.060533 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3155d3-9b1d-46a4-ba53-9a75e0b18d2c None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:14.065910 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff964b0-0c1f-4720-a9da-e9349641856c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:14.066804 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff964b0-0c1f-4720-a9da-e9349641856c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:14.067132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff964b0-0c1f-4720-a9da-e9349641856c admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:14.067412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff964b0-0c1f-4720-a9da-e9349641856c admin admin] PATH_INFO: `/v3/users/30cfb7257a0644a3a2c75bd2a0c43559` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:14.069142 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff964b0-0c1f-4720-a9da-e9349641856c admin admin] RBAC: Authorizing `identity:delete_user(user_id=30cfb7257a0644a3a2c75bd2a0c43559)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:14.072240 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff964b0-0c1f-4720-a9da-e9349641856c admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:14.332672 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45707df-1560-41b3-998a-0b9bd3791244 service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:14.344008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45707df-1560-41b3-998a-0b9bd3791244 service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:14.348658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfc99e-3d08-4204-885f-437e8fbad850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:14.349533 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfc99e-3d08-4204-885f-437e8fbad850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:14.353075 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfc99e-3d08-4204-885f-437e8fbad850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:14.353075 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfc99e-3d08-4204-885f-437e8fbad850 admin admin] PATH_INFO: `/v3/users/0051a1bd520e47229d232c7988b591bb` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:14.353075 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cfc99e-3d08-4204-885f-437e8fbad850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0051a1bd520e47229d232c7988b591bb)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:14.353361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cfc99e-3d08-4204-885f-437e8fbad850 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:14.359098 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:14.391305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-6ff964b0-0c1f-4720-a9da-e9349641856c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30cfb7257a0644a3a2c75bd2a0c43559'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:14.398875 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-6ff964b0-0c1f-4720-a9da-e9349641856c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30cfb7257a0644a3a2c75bd2a0c43559'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:14.399788 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-6ff964b0-0c1f-4720-a9da-e9349641856c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30cfb7257a0644a3a2c75bd2a0c43559'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:14.400391 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-6ff964b0-0c1f-4720-a9da-e9349641856c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30cfb7257a0644a3a2c75bd2a0c43559'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:14.412938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-6ff964b0-0c1f-4720-a9da-e9349641856c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30cfb7257a0644a3a2c75bd2a0c43559'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:14.504827 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 981/1964] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:25:13 2022] DELETE /identity/v3/users/30cfb7257a0644a3a2c75bd2a0c43559 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:14.511581 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279e1e1-145c-4511-9903-daa0423ac8ff None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:14.511870 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279e1e1-145c-4511-9903-daa0423ac8ff None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:14.512151 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279e1e1-145c-4511-9903-daa0423ac8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:14.530231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-4279e1e1-145c-4511-9903-daa0423ac8ff None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:14.532188 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-4279e1e1-145c-4511-9903-daa0423ac8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:14.590716 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 982/1965] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:25:14.601085 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279e1e1-145c-4511-9903-daa0423ac8ff None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:14.602429 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-4279e1e1-145c-4511-9903-daa0423ac8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:14.615085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-95cfc99e-3d08-4204-885f-437e8fbad850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0051a1bd520e47229d232c7988b591bb'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:14.615695 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:14.621879 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-95cfc99e-3d08-4204-885f-437e8fbad850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0051a1bd520e47229d232c7988b591bb'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:14.622561 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-95cfc99e-3d08-4204-885f-437e8fbad850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0051a1bd520e47229d232c7988b591bb'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:14.623208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-95cfc99e-3d08-4204-885f-437e8fbad850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0051a1bd520e47229d232c7988b591bb'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:14.636953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-95cfc99e-3d08-4204-885f-437e8fbad850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0051a1bd520e47229d232c7988b591bb'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:14.670003 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279e1e1-145c-4511-9903-daa0423ac8ff None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:14.673732 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34febe8-f6ef-4b65-a7a0-1d96f4303d98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:14.674394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34febe8-f6ef-4b65-a7a0-1d96f4303d98 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:14.674678 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34febe8-f6ef-4b65-a7a0-1d96f4303d98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:14.674983 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34febe8-f6ef-4b65-a7a0-1d96f4303d98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:14.676765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-f34febe8-f6ef-4b65-a7a0-1d96f4303d98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:14.688823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34febe8-f6ef-4b65-a7a0-1d96f4303d98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:14.690777 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34febe8-f6ef-4b65-a7a0-1d96f4303d98 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:14.700011 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 983/1966] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:25:14.787903 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 984/1967] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:25:14 2022] DELETE /identity/v3/users/0051a1bd520e47229d232c7988b591bb => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:25:14.795555 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5877c1f-02fb-4f68-a7a2-7aae372a30a9 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:14.795857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5877c1f-02fb-4f68-a7a2-7aae372a30a9 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:14.796160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5877c1f-02fb-4f68-a7a2-7aae372a30a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:14.814784 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-b5877c1f-02fb-4f68-a7a2-7aae372a30a9 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:14.816033 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-b5877c1f-02fb-4f68-a7a2-7aae372a30a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:14.879283 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 984/1968] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:14.895953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfc99e-3d08-4204-885f-437e8fbad850 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:14.897428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-95cfc99e-3d08-4204-885f-437e8fbad850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:14.966182 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfc99e-3d08-4204-885f-437e8fbad850 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:14.972321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7344bd-d0ce-443f-b035-970693f575a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:14.973147 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7344bd-d0ce-443f-b035-970693f575a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:14.973543 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7344bd-d0ce-443f-b035-970693f575a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:14.974079 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7344bd-d0ce-443f-b035-970693f575a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:14.986976 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7344bd-d0ce-443f-b035-970693f575a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:14.988687 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7344bd-d0ce-443f-b035-970693f575a8 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:15.003603 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 985/1969] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:25:15.231401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5877c1f-02fb-4f68-a7a2-7aae372a30a9 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:15.232921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-b5877c1f-02fb-4f68-a7a2-7aae372a30a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:15.245897 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5877c1f-02fb-4f68-a7a2-7aae372a30a9 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:15.251451 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4ab6f-ba6b-4c37-a2cd-c599aa3d9fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:15.252455 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4ab6f-ba6b-4c37-a2cd-c599aa3d9fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:15.252807 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4ab6f-ba6b-4c37-a2cd-c599aa3d9fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:15.253179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4ab6f-ba6b-4c37-a2cd-c599aa3d9fbc admin admin] PATH_INFO: `/v3/projects/fde682346eb14236967f36a6aea93686` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:15.254473 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4ab6f-ba6b-4c37-a2cd-c599aa3d9fbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=fde682346eb14236967f36a6aea93686)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:15.270859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4ab6f-ba6b-4c37-a2cd-c599aa3d9fbc admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:15.432400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-55c4ab6f-ba6b-4c37-a2cd-c599aa3d9fbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fde682346eb14236967f36a6aea93686', 'request_id': 'req-55c4ab6f-ba6b-4c37-a2cd-c599aa3d9fbc'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:15.440553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-55c4ab6f-ba6b-4c37-a2cd-c599aa3d9fbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fde682346eb14236967f36a6aea93686', 'request_id': 'req-55c4ab6f-ba6b-4c37-a2cd-c599aa3d9fbc'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:15.532397 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-55c4ab6f-ba6b-4c37-a2cd-c599aa3d9fbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fde682346eb14236967f36a6aea93686', 'request_id': 'req-55c4ab6f-ba6b-4c37-a2cd-c599aa3d9fbc'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:15.552378 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d486482-ecb9-443d-9f8f-58dcb81bb993 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:15.552876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d486482-ecb9-443d-9f8f-58dcb81bb993 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:15.553050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d486482-ecb9-443d-9f8f-58dcb81bb993 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:15.554664 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 986/1970] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jun 23 09:25:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:25:15.558579 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-55c4ab6f-ba6b-4c37-a2cd-c599aa3d9fbc admin admin] The token cache is being invalidate because project fde682346eb14236967f36a6aea93686 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:25:15.559136 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-55c4ab6f-ba6b-4c37-a2cd-c599aa3d9fbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:15.559890 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-55c4ab6f-ba6b-4c37-a2cd-c599aa3d9fbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:15.561475 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 985/1971] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:25:15 2022] DELETE /identity/v3/projects/fde682346eb14236967f36a6aea93686 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:15.572210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d486482-ecb9-443d-9f8f-58dcb81bb993 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:15.572210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-9d486482-ecb9-443d-9f8f-58dcb81bb993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:15.578627 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e10ab-75ed-4ee5-959d-5a3410aa6e3f None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:15.582182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e10ab-75ed-4ee5-959d-5a3410aa6e3f None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:15.582598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e10ab-75ed-4ee5-959d-5a3410aa6e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:15.626168 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-860e10ab-75ed-4ee5-959d-5a3410aa6e3f None None] MFA Rules not processed for user `98a63b20ceca40e6bceb56eda8ae9686`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:15.626168 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-860e10ab-75ed-4ee5-959d-5a3410aa6e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:15.657229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:15.661751 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d486482-ecb9-443d-9f8f-58dcb81bb993 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:15.666397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38827b50-1fe0-439a-b0ad-9b5ed160acad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:15.667309 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38827b50-1fe0-439a-b0ad-9b5ed160acad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:15.667602 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38827b50-1fe0-439a-b0ad-9b5ed160acad admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:15.667886 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38827b50-1fe0-439a-b0ad-9b5ed160acad admin admin] PATH_INFO: `/v3/users/036b13ae809d404c88020fd383cfde3d` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:15.669103 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38827b50-1fe0-439a-b0ad-9b5ed160acad admin admin] RBAC: Authorizing `identity:delete_user(user_id=036b13ae809d404c88020fd383cfde3d)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:15.671064 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38827b50-1fe0-439a-b0ad-9b5ed160acad admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:15.691970 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 986/1972] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jun 23 09:25:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:15.693346 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e10ab-75ed-4ee5-959d-5a3410aa6e3f None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:15.699556 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-860e10ab-75ed-4ee5-959d-5a3410aa6e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:15.713751 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e10ab-75ed-4ee5-959d-5a3410aa6e3f None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:15.718957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008e516-2aca-47c8-bb35-0dcdefa25b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:15.719888 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008e516-2aca-47c8-bb35-0dcdefa25b0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:15.720181 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008e516-2aca-47c8-bb35-0dcdefa25b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:15.720746 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008e516-2aca-47c8-bb35-0dcdefa25b0d admin admin] PATH_INFO: `/v3/projects/56c1fd17565b4ec283f7a86ae7bd71a4` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:15.722574 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008e516-2aca-47c8-bb35-0dcdefa25b0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=56c1fd17565b4ec283f7a86ae7bd71a4)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:15.726229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008e516-2aca-47c8-bb35-0dcdefa25b0d admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:15.886940 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:15.900072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-38827b50-1fe0-439a-b0ad-9b5ed160acad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '036b13ae809d404c88020fd383cfde3d'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:15.900491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-6008e516-2aca-47c8-bb35-0dcdefa25b0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56c1fd17565b4ec283f7a86ae7bd71a4', 'request_id': 'req-6008e516-2aca-47c8-bb35-0dcdefa25b0d'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:15.907516 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-6008e516-2aca-47c8-bb35-0dcdefa25b0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56c1fd17565b4ec283f7a86ae7bd71a4', 'request_id': 'req-6008e516-2aca-47c8-bb35-0dcdefa25b0d'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:15.907866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-38827b50-1fe0-439a-b0ad-9b5ed160acad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '036b13ae809d404c88020fd383cfde3d'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:15.908612 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-38827b50-1fe0-439a-b0ad-9b5ed160acad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '036b13ae809d404c88020fd383cfde3d'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:15.909322 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-38827b50-1fe0-439a-b0ad-9b5ed160acad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '036b13ae809d404c88020fd383cfde3d'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:15.921043 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-38827b50-1fe0-439a-b0ad-9b5ed160acad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '036b13ae809d404c88020fd383cfde3d'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:16.023908 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-6008e516-2aca-47c8-bb35-0dcdefa25b0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56c1fd17565b4ec283f7a86ae7bd71a4', 'request_id': 'req-6008e516-2aca-47c8-bb35-0dcdefa25b0d'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:16.036105 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-6008e516-2aca-47c8-bb35-0dcdefa25b0d admin admin] The token cache is being invalidate because project 56c1fd17565b4ec283f7a86ae7bd71a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:25:16.036396 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-6008e516-2aca-47c8-bb35-0dcdefa25b0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:16.037640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-6008e516-2aca-47c8-bb35-0dcdefa25b0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:16.039307 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 987/1973] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:25:15 2022] DELETE /identity/v3/projects/56c1fd17565b4ec283f7a86ae7bd71a4 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:16.042554 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008e516-2aca-47c8-bb35-0dcdefa25b0d admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:16.042554 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-6008e516-2aca-47c8-bb35-0dcdefa25b0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:16.114143 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008e516-2aca-47c8-bb35-0dcdefa25b0d admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:16.119418 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce6991-4e0e-4719-b8cd-f174e6dcb4ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63b20ceca40e6bceb56eda8ae9686', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:16.120079 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce6991-4e0e-4719-b8cd-f174e6dcb4ee service placement] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:16.120367 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce6991-4e0e-4719-b8cd-f174e6dcb4ee service placement] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:16.120646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce6991-4e0e-4719-b8cd-f174e6dcb4ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:16.122480 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-27ce6991-4e0e-4719-b8cd-f174e6dcb4ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:16.134106 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce6991-4e0e-4719-b8cd-f174e6dcb4ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:16.135766 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce6991-4e0e-4719-b8cd-f174e6dcb4ee service placement] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:16.146270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 988/1974] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:25:16.147705 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c144ded-d7bb-4973-b4c4-2183b5665922 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:16.148019 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c144ded-d7bb-4973-b4c4-2183b5665922 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:16.148450 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c144ded-d7bb-4973-b4c4-2183b5665922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:16.179972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 987/1975] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:25:15 2022] DELETE /identity/v3/users/036b13ae809d404c88020fd383cfde3d => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:16.191706 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-7c144ded-d7bb-4973-b4c4-2183b5665922 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:16.193490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-7c144ded-d7bb-4973-b4c4-2183b5665922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:16.284632 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 989/1976] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:16.293634 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b85e42-0ad0-4964-b77c-7852be588339 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:16.294561 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b85e42-0ad0-4964-b77c-7852be588339 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:16.294561 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b85e42-0ad0-4964-b77c-7852be588339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:16.318270 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-74b85e42-0ad0-4964-b77c-7852be588339 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:16.319739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-74b85e42-0ad0-4964-b77c-7852be588339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:16.332237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 988/1977] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:16.336757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c144ded-d7bb-4973-b4c4-2183b5665922 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:16.349568 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c144ded-d7bb-4973-b4c4-2183b5665922 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:16.360088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc93455-ebce-4b34-9b24-762dd30efab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:16.362066 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc93455-ebce-4b34-9b24-762dd30efab1 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:16.362511 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc93455-ebce-4b34-9b24-762dd30efab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:16.362897 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc93455-ebce-4b34-9b24-762dd30efab1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:16.363961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc93455-ebce-4b34-9b24-762dd30efab1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:25:16.364530 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc93455-ebce-4b34-9b24-762dd30efab1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:16.368644 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc93455-ebce-4b34-9b24-762dd30efab1 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:16.392919 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b85e42-0ad0-4964-b77c-7852be588339 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:16.393124 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 990/1978] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:25:16.410261 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b85e42-0ad0-4964-b77c-7852be588339 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:16.415476 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6d0c3-3159-45f5-b1ef-d7d3abaf0bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:16.428869 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6d0c3-3159-45f5-b1ef-d7d3abaf0bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:16.430994 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6d0c3-3159-45f5-b1ef-d7d3abaf0bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:16.431925 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6d0c3-3159-45f5-b1ef-d7d3abaf0bb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:16.432705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd6d0c3-3159-45f5-b1ef-d7d3abaf0bb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:16.446641 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd6d0c3-3159-45f5-b1ef-d7d3abaf0bb5 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:16.460310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6311ba9-5c3f-42b1-93ab-aac463e00e15 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:16.460604 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6311ba9-5c3f-42b1-93ab-aac463e00e15 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:16.460894 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6311ba9-5c3f-42b1-93ab-aac463e00e15 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:16.462443 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 991/1979] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jun 23 09:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 09:25:16.501757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fc9c8-82b4-49da-9db4-f8c1d3bf9c1e None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:16.502049 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fc9c8-82b4-49da-9db4-f8c1d3bf9c1e None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:16.502329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fc9c8-82b4-49da-9db4-f8c1d3bf9c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:16.521138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-f68fc9c8-82b4-49da-9db4-f8c1d3bf9c1e None None] MFA Rules not processed for user `98a63b20ceca40e6bceb56eda8ae9686`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:16.522382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-f68fc9c8-82b4-49da-9db4-f8c1d3bf9c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:16.589847 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 989/1980] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:25:16 2022] POST /identity/v3/projects => generated 361 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:25:16.595021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6d0c3-3159-45f5-b1ef-d7d3abaf0bb5 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:16.641383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 992/1981] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jun 23 09:25:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:25:16.665730 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fc9c8-82b4-49da-9db4-f8c1d3bf9c1e None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:16.681545 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fc9c8-82b4-49da-9db4-f8c1d3bf9c1e None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:16.726570 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6d0c3-3159-45f5-b1ef-d7d3abaf0bb5 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:16.735557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72472dae-ba56-41e6-8354-98bf3281e37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:16.739219 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72472dae-ba56-41e6-8354-98bf3281e37c admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:16.743600 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72472dae-ba56-41e6-8354-98bf3281e37c admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:16.743600 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72472dae-ba56-41e6-8354-98bf3281e37c admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:16.743600 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72472dae-ba56-41e6-8354-98bf3281e37c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:16.747678 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72472dae-ba56-41e6-8354-98bf3281e37c admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:16.788927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea6b33-6559-4d31-9666-c93a282f7344 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63b20ceca40e6bceb56eda8ae9686', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:16.789602 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea6b33-6559-4d31-9666-c93a282f7344 service placement] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:16.789892 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea6b33-6559-4d31-9666-c93a282f7344 service placement] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:16.790172 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea6b33-6559-4d31-9666-c93a282f7344 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:16.806845 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea6b33-6559-4d31-9666-c93a282f7344 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:16.806845 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea6b33-6559-4d31-9666-c93a282f7344 service placement] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:16.812114 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 993/1982] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:25:16.812943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea6b33-6559-4d31-9666-c93a282f7344 service placement] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:16.814281 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5aea6b33-6559-4d31-9666-c93a282f7344 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:16.829168 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea6b33-6559-4d31-9666-c93a282f7344 service placement] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:16.835104 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fec85d-9e16-4dd0-acee-8be4940175ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:16.837666 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fec85d-9e16-4dd0-acee-8be4940175ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:16.837666 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fec85d-9e16-4dd0-acee-8be4940175ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:16.837666 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fec85d-9e16-4dd0-acee-8be4940175ce admin admin] PATH_INFO: `/v3/users/7e9ecac5434e4d40806f57d53256f7fa` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:16.837666 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fec85d-9e16-4dd0-acee-8be4940175ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e9ecac5434e4d40806f57d53256f7fa)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:16.839317 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fec85d-9e16-4dd0-acee-8be4940175ce admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:16.906673 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 990/1983] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:25:16 2022] POST /identity/v3/users => generated 281 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:25:16.909017 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72472dae-ba56-41e6-8354-98bf3281e37c admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:16.910241 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:16.921858 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72472dae-ba56-41e6-8354-98bf3281e37c admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:16.927223 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a8f55-04e6-4f74-ab95-28cfe544bbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:16.927950 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a8f55-04e6-4f74-ab95-28cfe544bbd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:16.928196 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a8f55-04e6-4f74-ab95-28cfe544bbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:16.928472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a8f55-04e6-4f74-ab95-28cfe544bbd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:16.929217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a8f55-04e6-4f74-ab95-28cfe544bbd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:16.931231 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a8f55-04e6-4f74-ab95-28cfe544bbd3 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:16.942789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 991/1984] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:25:16.943640 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a8f55-04e6-4f74-ab95-28cfe544bbd3 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:16.947021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e10a8f55-04e6-4f74-ab95-28cfe544bbd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:16.960763 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a8f55-04e6-4f74-ab95-28cfe544bbd3 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:16.965928 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38045089-9f83-4d47-ae14-3d1a9194858c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:16.969765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38045089-9f83-4d47-ae14-3d1a9194858c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:16.969765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38045089-9f83-4d47-ae14-3d1a9194858c admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:16.969765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38045089-9f83-4d47-ae14-3d1a9194858c admin admin] PATH_INFO: `/v3/projects/2d694507867c4c46940a94b1262c2aaf` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:16.969765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38045089-9f83-4d47-ae14-3d1a9194858c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d694507867c4c46940a94b1262c2aaf)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:16.971432 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38045089-9f83-4d47-ae14-3d1a9194858c admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:17.074328 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-82fec85d-9e16-4dd0-acee-8be4940175ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e9ecac5434e4d40806f57d53256f7fa'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:17.083195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-82fec85d-9e16-4dd0-acee-8be4940175ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e9ecac5434e4d40806f57d53256f7fa'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:17.083941 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-82fec85d-9e16-4dd0-acee-8be4940175ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e9ecac5434e4d40806f57d53256f7fa'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:17.084926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-82fec85d-9e16-4dd0-acee-8be4940175ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e9ecac5434e4d40806f57d53256f7fa'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:17.105192 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-82fec85d-9e16-4dd0-acee-8be4940175ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e9ecac5434e4d40806f57d53256f7fa'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:17.127112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-38045089-9f83-4d47-ae14-3d1a9194858c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d694507867c4c46940a94b1262c2aaf', 'request_id': 'req-38045089-9f83-4d47-ae14-3d1a9194858c'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:17.133837 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-38045089-9f83-4d47-ae14-3d1a9194858c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d694507867c4c46940a94b1262c2aaf', 'request_id': 'req-38045089-9f83-4d47-ae14-3d1a9194858c'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:17.139218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-38045089-9f83-4d47-ae14-3d1a9194858c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d694507867c4c46940a94b1262c2aaf', 'request_id': 'req-38045089-9f83-4d47-ae14-3d1a9194858c'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:17.297267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 994/1985] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:25:16 2022] DELETE /identity/v3/users/7e9ecac5434e4d40806f57d53256f7fa => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:17.297267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fec85d-9e16-4dd0-acee-8be4940175ce admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:17.297267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-82fec85d-9e16-4dd0-acee-8be4940175ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:17.398994 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fec85d-9e16-4dd0-acee-8be4940175ce admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:17.403819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3001ec-3007-463e-91b8-78282b2dc063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:17.404795 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3001ec-3007-463e-91b8-78282b2dc063 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:17.404929 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3001ec-3007-463e-91b8-78282b2dc063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:17.405096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3001ec-3007-463e-91b8-78282b2dc063 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:17.405984 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3001ec-3007-463e-91b8-78282b2dc063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:17.407781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3001ec-3007-463e-91b8-78282b2dc063 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:17.419183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 995/1986] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:25:17.423261 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3001ec-3007-463e-91b8-78282b2dc063 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:17.434896 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-38045089-9f83-4d47-ae14-3d1a9194858c admin admin] The token cache is being invalidate because project 2d694507867c4c46940a94b1262c2aaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:25:17.434896 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-38045089-9f83-4d47-ae14-3d1a9194858c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:17.434896 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-38045089-9f83-4d47-ae14-3d1a9194858c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:17.434896 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 992/1987] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:25:16 2022] DELETE /identity/v3/projects/2d694507867c4c46940a94b1262c2aaf => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:17.440657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3001ec-3007-463e-91b8-78282b2dc063 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:17.441756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a1e9c-bde9-45da-8e33-9376a849fdbe None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:17.442244 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a1e9c-bde9-45da-8e33-9376a849fdbe None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:17.442726 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a1e9c-bde9-45da-8e33-9376a849fdbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:17.449868 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef83d8-09e8-4cc9-a155-0817738f9389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:17.450785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef83d8-09e8-4cc9-a155-0817738f9389 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:17.451110 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef83d8-09e8-4cc9-a155-0817738f9389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:17.451917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef83d8-09e8-4cc9-a155-0817738f9389 admin admin] PATH_INFO: `/v3/projects/033e7a9eee0149388e1c0815ba8f3ae1/users/3c54fb85f05047b6a8b3ae0be7cc64f2/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:17.460639 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:17.474115 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef83d8-09e8-4cc9-a155-0817738f9389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=033e7a9eee0149388e1c0815ba8f3ae1, user_id=3c54fb85f05047b6a8b3ae0be7cc64f2, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:17.477428 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef83d8-09e8-4cc9-a155-0817738f9389 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:17.488825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-300a1e9c-bde9-45da-8e33-9376a849fdbe None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:17.488825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-300a1e9c-bde9-45da-8e33-9376a849fdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:17.530418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 993/1988] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:17.538940 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b15b6-5df9-4f1d-a756-4f77eb9f1c16 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:17.541126 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b15b6-5df9-4f1d-a756-4f77eb9f1c16 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:17.541463 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b15b6-5df9-4f1d-a756-4f77eb9f1c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:17.577321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-e87b15b6-5df9-4f1d-a756-4f77eb9f1c16 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:17.578843 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e87b15b6-5df9-4f1d-a756-4f77eb9f1c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:17.614413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 994/1989] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:17.615520 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b15b6-5df9-4f1d-a756-4f77eb9f1c16 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:17.620219 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e87b15b6-5df9-4f1d-a756-4f77eb9f1c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:17.622392 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 996/1990] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:25:17 2022] PUT /identity/v3/projects/033e7a9eee0149388e1c0815ba8f3ae1/users/3c54fb85f05047b6a8b3ae0be7cc64f2/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:25:17.623215 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef83d8-09e8-4cc9-a155-0817738f9389 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:17.636788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b15b6-5df9-4f1d-a756-4f77eb9f1c16 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:17.737052 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cdf80e-16ec-41f4-8100-afb9fa3b5bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:17.737803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cdf80e-16ec-41f4-8100-afb9fa3b5bf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:17.738206 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cdf80e-16ec-41f4-8100-afb9fa3b5bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:17.738716 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cdf80e-16ec-41f4-8100-afb9fa3b5bf2 admin admin] PATH_INFO: `/v3/users/911ff3294dfe46509f74c3cb8c558863` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:17.741410 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cdf80e-16ec-41f4-8100-afb9fa3b5bf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=911ff3294dfe46509f74c3cb8c558863)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:17.743849 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cdf80e-16ec-41f4-8100-afb9fa3b5bf2 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:17.749017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef83d8-09e8-4cc9-a155-0817738f9389 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:17.762392 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874afee8-25f5-493d-bf7b-272e6616efb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:17.763114 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874afee8-25f5-493d-bf7b-272e6616efb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:17.763403 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874afee8-25f5-493d-bf7b-272e6616efb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:17.763684 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874afee8-25f5-493d-bf7b-272e6616efb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:17.764424 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874afee8-25f5-493d-bf7b-272e6616efb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:25:17.765724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874afee8-25f5-493d-bf7b-272e6616efb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:17.767438 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874afee8-25f5-493d-bf7b-272e6616efb8 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:17.780668 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 997/1991] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:25:17.783382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289a44b-33e2-4fd3-9aab-72ec9798dc6c None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:17.783672 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289a44b-33e2-4fd3-9aab-72ec9798dc6c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:17.783951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289a44b-33e2-4fd3-9aab-72ec9798dc6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:17.817084 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-d289a44b-33e2-4fd3-9aab-72ec9798dc6c None None] MFA Rules not processed for user `3c54fb85f05047b6a8b3ae0be7cc64f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:17.818482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d289a44b-33e2-4fd3-9aab-72ec9798dc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:17.894894 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 998/1992] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:17.895909 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289a44b-33e2-4fd3-9aab-72ec9798dc6c None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:17.897154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d289a44b-33e2-4fd3-9aab-72ec9798dc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:17.910251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289a44b-33e2-4fd3-9aab-72ec9798dc6c None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:17.915738 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc15f9b-8fe4-4cc8-ace4-3ca74dd96c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:17.916659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc15f9b-8fe4-4cc8-ace4-3ca74dd96c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:17.916957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc15f9b-8fe4-4cc8-ace4-3ca74dd96c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:17.917267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc15f9b-8fe4-4cc8-ace4-3ca74dd96c9d admin admin] PATH_INFO: `/v3/projects/5a7afaa88f93446884e96bb39142d3b4` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:17.918466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc15f9b-8fe4-4cc8-ace4-3ca74dd96c9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a7afaa88f93446884e96bb39142d3b4)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:17.920513 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc15f9b-8fe4-4cc8-ace4-3ca74dd96c9d admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:17.992249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-34cdf80e-16ec-41f4-8100-afb9fa3b5bf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '911ff3294dfe46509f74c3cb8c558863'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:17.999176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-34cdf80e-16ec-41f4-8100-afb9fa3b5bf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '911ff3294dfe46509f74c3cb8c558863'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:18.000357 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-34cdf80e-16ec-41f4-8100-afb9fa3b5bf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '911ff3294dfe46509f74c3cb8c558863'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:18.001001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-34cdf80e-16ec-41f4-8100-afb9fa3b5bf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '911ff3294dfe46509f74c3cb8c558863'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:18.012537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-34cdf80e-16ec-41f4-8100-afb9fa3b5bf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '911ff3294dfe46509f74c3cb8c558863'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:18.102868 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-acc15f9b-8fe4-4cc8-ace4-3ca74dd96c9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a7afaa88f93446884e96bb39142d3b4', 'request_id': 'req-acc15f9b-8fe4-4cc8-ace4-3ca74dd96c9d'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:18.109393 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-acc15f9b-8fe4-4cc8-ace4-3ca74dd96c9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a7afaa88f93446884e96bb39142d3b4', 'request_id': 'req-acc15f9b-8fe4-4cc8-ace4-3ca74dd96c9d'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:18.124142 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 995/1993] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:25:17 2022] DELETE /identity/v3/users/911ff3294dfe46509f74c3cb8c558863 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:18.125600 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cdf80e-16ec-41f4-8100-afb9fa3b5bf2 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:18.127145 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-34cdf80e-16ec-41f4-8100-afb9fa3b5bf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:18.183401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-acc15f9b-8fe4-4cc8-ace4-3ca74dd96c9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a7afaa88f93446884e96bb39142d3b4', 'request_id': 'req-acc15f9b-8fe4-4cc8-ace4-3ca74dd96c9d'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:18.195831 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-acc15f9b-8fe4-4cc8-ace4-3ca74dd96c9d admin admin] The token cache is being invalidate because project 5a7afaa88f93446884e96bb39142d3b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:25:18.196127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-acc15f9b-8fe4-4cc8-ace4-3ca74dd96c9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:18.197027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-acc15f9b-8fe4-4cc8-ace4-3ca74dd96c9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:18.198634 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 999/1994] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:25:17 2022] DELETE /identity/v3/projects/5a7afaa88f93446884e96bb39142d3b4 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:18.199915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937cb9c-48cc-4e3d-8143-ae3230b1a936 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:18.200206 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937cb9c-48cc-4e3d-8143-ae3230b1a936 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:18.200491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937cb9c-48cc-4e3d-8143-ae3230b1a936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:18.212449 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cdf80e-16ec-41f4-8100-afb9fa3b5bf2 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:18.217521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc71cc-4146-4648-b800-d891e370e9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:18.218223 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc71cc-4146-4648-b800-d891e370e9ef admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:18.218557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc71cc-4146-4648-b800-d891e370e9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:18.218836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc71cc-4146-4648-b800-d891e370e9ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:18.219264 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-4937cb9c-48cc-4e3d-8143-ae3230b1a936 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:18.219612 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc71cc-4146-4648-b800-d891e370e9ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:18.220489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-4937cb9c-48cc-4e3d-8143-ae3230b1a936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:18.222330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc71cc-4146-4648-b800-d891e370e9ef admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:18.229776 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1000/1995] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:18.231982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937cb9c-48cc-4e3d-8143-ae3230b1a936 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:18.236719 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-4937cb9c-48cc-4e3d-8143-ae3230b1a936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:18.297119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937cb9c-48cc-4e3d-8143-ae3230b1a936 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:18.300817 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7c99b-351c-4841-ad35-0625d9137957 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:18.301470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7c99b-351c-4841-ad35-0625d9137957 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:18.301757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7c99b-351c-4841-ad35-0625d9137957 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:18.302037 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7c99b-351c-4841-ad35-0625d9137957 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:18.303854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-e4b7c99b-351c-4841-ad35-0625d9137957 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:18.360246 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 996/1996] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:25:18 2022] POST /identity/v3/projects => generated 373 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:25:18.360246 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc71cc-4146-4648-b800-d891e370e9ef admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:18.361990 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-69fc71cc-4146-4648-b800-d891e370e9ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:18.366846 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7c99b-351c-4841-ad35-0625d9137957 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:18.368324 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7c99b-351c-4841-ad35-0625d9137957 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:18.379545 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1001/1997] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:25:18.380808 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7c99b-351c-4841-ad35-0625d9137957 service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:18.384216 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-e4b7c99b-351c-4841-ad35-0625d9137957 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:18.432482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc71cc-4146-4648-b800-d891e370e9ef admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:18.438342 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a500e9f-8940-4ae9-b595-b1c7d5966e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:18.439283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a500e9f-8940-4ae9-b595-b1c7d5966e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:18.439592 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a500e9f-8940-4ae9-b595-b1c7d5966e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:18.439869 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a500e9f-8940-4ae9-b595-b1c7d5966e5e admin admin] PATH_INFO: `/v3/users/ca68ffe4044d44b8bdb7c326971d4b12` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:18.442216 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a500e9f-8940-4ae9-b595-b1c7d5966e5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca68ffe4044d44b8bdb7c326971d4b12)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:18.444290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a500e9f-8940-4ae9-b595-b1c7d5966e5e admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:18.446134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7c99b-351c-4841-ad35-0625d9137957 service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:18.498761 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc05c1c-1fda-4f21-8164-1491ca0e201f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:18.499452 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc05c1c-1fda-4f21-8164-1491ca0e201f service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:18.500291 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc05c1c-1fda-4f21-8164-1491ca0e201f service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:18.500575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc05c1c-1fda-4f21-8164-1491ca0e201f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:18.509851 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc05c1c-1fda-4f21-8164-1491ca0e201f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:18.511338 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc05c1c-1fda-4f21-8164-1491ca0e201f service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:18.517074 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:18.525131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1002/1998] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:25:18.525778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc05c1c-1fda-4f21-8164-1491ca0e201f service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:18.527389 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-6dc05c1c-1fda-4f21-8164-1491ca0e201f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:18.540071 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc05c1c-1fda-4f21-8164-1491ca0e201f service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:18.545591 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1a322-750c-4a1f-8d1c-5428cae431a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:18.546298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1a322-750c-4a1f-8d1c-5428cae431a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:18.550547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1a322-750c-4a1f-8d1c-5428cae431a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:18.550547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1a322-750c-4a1f-8d1c-5428cae431a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:18.550547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1a322-750c-4a1f-8d1c-5428cae431a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:18.550547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1a322-750c-4a1f-8d1c-5428cae431a1 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:18.709734 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1003/1999] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:25:18 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:25:18.713682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1a322-750c-4a1f-8d1c-5428cae431a1 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:18.727880 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1a322-750c-4a1f-8d1c-5428cae431a1 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:18.727985 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-1a500e9f-8940-4ae9-b595-b1c7d5966e5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca68ffe4044d44b8bdb7c326971d4b12'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:18.731384 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95e715-6f4f-48db-b9d7-14e1647aea7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:18.732233 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95e715-6f4f-48db-b9d7-14e1647aea7c admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:18.732530 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95e715-6f4f-48db-b9d7-14e1647aea7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:18.732813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95e715-6f4f-48db-b9d7-14e1647aea7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:18.734728 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c95e715-6f4f-48db-b9d7-14e1647aea7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:18.735778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-1a500e9f-8940-4ae9-b595-b1c7d5966e5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca68ffe4044d44b8bdb7c326971d4b12'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:18.736557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-1a500e9f-8940-4ae9-b595-b1c7d5966e5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca68ffe4044d44b8bdb7c326971d4b12'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:18.736717 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c95e715-6f4f-48db-b9d7-14e1647aea7c admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:18.737287 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-1a500e9f-8940-4ae9-b595-b1c7d5966e5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca68ffe4044d44b8bdb7c326971d4b12'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:18.752155 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-1a500e9f-8940-4ae9-b595-b1c7d5966e5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca68ffe4044d44b8bdb7c326971d4b12'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:18.769924 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1004/2000] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:25:18.771788 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95e715-6f4f-48db-b9d7-14e1647aea7c admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:18.773518 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-6c95e715-6f4f-48db-b9d7-14e1647aea7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:18.802612 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95e715-6f4f-48db-b9d7-14e1647aea7c admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:18.807437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09aa908-f871-4fc0-9fc8-2a1b50c56be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:18.808114 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09aa908-f871-4fc0-9fc8-2a1b50c56be8 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:18.808403 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09aa908-f871-4fc0-9fc8-2a1b50c56be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:18.808737 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09aa908-f871-4fc0-9fc8-2a1b50c56be8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:18.809472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09aa908-f871-4fc0-9fc8-2a1b50c56be8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:18.811165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09aa908-f871-4fc0-9fc8-2a1b50c56be8 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:18.822235 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1005/2001] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:25:18.826309 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09aa908-f871-4fc0-9fc8-2a1b50c56be8 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:18.836020 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09aa908-f871-4fc0-9fc8-2a1b50c56be8 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:18.840716 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7565c-cbc6-456a-a5ff-25dbb3cb9b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:18.841559 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7565c-cbc6-456a-a5ff-25dbb3cb9b05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:18.841850 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7565c-cbc6-456a-a5ff-25dbb3cb9b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:18.842127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7565c-cbc6-456a-a5ff-25dbb3cb9b05 admin admin] PATH_INFO: `/v3/projects/d8faa464772d41a9b71e36e2c0c5d57f/users/431570aae7d64813a4d6899f39270846/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:18.859920 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f7565c-cbc6-456a-a5ff-25dbb3cb9b05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8faa464772d41a9b71e36e2c0c5d57f, user_id=431570aae7d64813a4d6899f39270846, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:18.862615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f7565c-cbc6-456a-a5ff-25dbb3cb9b05 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:18.945420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:18.951867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 997/2002] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:25:18 2022] DELETE /identity/v3/users/ca68ffe4044d44b8bdb7c326971d4b12 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:19.010087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1006/2003] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:25:18 2022] PUT /identity/v3/projects/d8faa464772d41a9b71e36e2c0c5d57f/users/431570aae7d64813a4d6899f39270846/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:25:19.011973 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55285665-7beb-4f83-a26e-213e78c065be None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:19.012337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55285665-7beb-4f83-a26e-213e78c065be None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:19.012683 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55285665-7beb-4f83-a26e-213e78c065be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:19.051508 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-55285665-7beb-4f83-a26e-213e78c065be None None] MFA Rules not processed for user `431570aae7d64813a4d6899f39270846`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:19.052905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-55285665-7beb-4f83-a26e-213e78c065be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:19.085602 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7565c-cbc6-456a-a5ff-25dbb3cb9b05 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:19.087925 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-91f7565c-cbc6-456a-a5ff-25dbb3cb9b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:19.246590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 998/2004] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:19.246590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55285665-7beb-4f83-a26e-213e78c065be None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:19.246590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-55285665-7beb-4f83-a26e-213e78c065be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:19.248060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7565c-cbc6-456a-a5ff-25dbb3cb9b05 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:19.263304 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6a452-7abc-4529-b25a-89fc8b1ca027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:19.264190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6a452-7abc-4529-b25a-89fc8b1ca027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:19.264477 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6a452-7abc-4529-b25a-89fc8b1ca027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:19.264785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6a452-7abc-4529-b25a-89fc8b1ca027 admin admin] PATH_INFO: `/v3/projects/79e51d48f9dc4c04b00dc0b5da9ded58` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:19.271815 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f6a452-7abc-4529-b25a-89fc8b1ca027 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79e51d48f9dc4c04b00dc0b5da9ded58)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:19.273874 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f6a452-7abc-4529-b25a-89fc8b1ca027 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:19.277791 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55285665-7beb-4f83-a26e-213e78c065be None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:19.287039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd01f57-75c9-408e-bdcb-9d29a645dbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:19.287947 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd01f57-75c9-408e-bdcb-9d29a645dbac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:19.288236 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd01f57-75c9-408e-bdcb-9d29a645dbac admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:19.288522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd01f57-75c9-408e-bdcb-9d29a645dbac admin admin] PATH_INFO: `/v3/users/2d0abc022f6f4a1d9a44375791e38663` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:19.292166 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd01f57-75c9-408e-bdcb-9d29a645dbac admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d0abc022f6f4a1d9a44375791e38663)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:19.294521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd01f57-75c9-408e-bdcb-9d29a645dbac admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:19.406799 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f1f6a452-7abc-4529-b25a-89fc8b1ca027 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79e51d48f9dc4c04b00dc0b5da9ded58', 'request_id': 'req-f1f6a452-7abc-4529-b25a-89fc8b1ca027'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:19.412887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f1f6a452-7abc-4529-b25a-89fc8b1ca027 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79e51d48f9dc4c04b00dc0b5da9ded58', 'request_id': 'req-f1f6a452-7abc-4529-b25a-89fc8b1ca027'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:19.507555 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-fbd01f57-75c9-408e-bdcb-9d29a645dbac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d0abc022f6f4a1d9a44375791e38663'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:19.515656 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-fbd01f57-75c9-408e-bdcb-9d29a645dbac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d0abc022f6f4a1d9a44375791e38663'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:19.516338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-fbd01f57-75c9-408e-bdcb-9d29a645dbac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d0abc022f6f4a1d9a44375791e38663'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:19.516948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-fbd01f57-75c9-408e-bdcb-9d29a645dbac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d0abc022f6f4a1d9a44375791e38663'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:19.529220 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-fbd01f57-75c9-408e-bdcb-9d29a645dbac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d0abc022f6f4a1d9a44375791e38663'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:19.555025 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f1f6a452-7abc-4529-b25a-89fc8b1ca027 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79e51d48f9dc4c04b00dc0b5da9ded58', 'request_id': 'req-f1f6a452-7abc-4529-b25a-89fc8b1ca027'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:19.566906 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f1f6a452-7abc-4529-b25a-89fc8b1ca027 admin admin] The token cache is being invalidate because project 79e51d48f9dc4c04b00dc0b5da9ded58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:25:19.567237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f1f6a452-7abc-4529-b25a-89fc8b1ca027 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:19.567915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f1f6a452-7abc-4529-b25a-89fc8b1ca027 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:19.569510 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1007/2005] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:25:19 2022] DELETE /identity/v3/projects/79e51d48f9dc4c04b00dc0b5da9ded58 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:19.570338 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6a452-7abc-4529-b25a-89fc8b1ca027 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:19.571637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-f1f6a452-7abc-4529-b25a-89fc8b1ca027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:19.584210 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:19.632779 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6a452-7abc-4529-b25a-89fc8b1ca027 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:19.636664 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f654539-b5fe-426e-be18-1cb8c9a3ac4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:19.637858 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f654539-b5fe-426e-be18-1cb8c9a3ac4f service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:19.637858 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f654539-b5fe-426e-be18-1cb8c9a3ac4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:19.638031 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f654539-b5fe-426e-be18-1cb8c9a3ac4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:19.639633 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5f654539-b5fe-426e-be18-1cb8c9a3ac4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:19.663353 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 999/2006] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:25:19 2022] DELETE /identity/v3/users/2d0abc022f6f4a1d9a44375791e38663 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:19.664288 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd01f57-75c9-408e-bdcb-9d29a645dbac admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:19.665863 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-fbd01f57-75c9-408e-bdcb-9d29a645dbac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:19.702451 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f654539-b5fe-426e-be18-1cb8c9a3ac4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:19.704534 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f654539-b5fe-426e-be18-1cb8c9a3ac4f service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:19.713996 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1008/2007] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:25:19.718319 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f654539-b5fe-426e-be18-1cb8c9a3ac4f service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:19.718319 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5f654539-b5fe-426e-be18-1cb8c9a3ac4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:19.740021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd01f57-75c9-408e-bdcb-9d29a645dbac admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:19.745288 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93e7f8c-8fd7-4698-80fd-6b98754cd837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:19.746198 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e7f8c-8fd7-4698-80fd-6b98754cd837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:19.746487 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e7f8c-8fd7-4698-80fd-6b98754cd837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:19.750834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e7f8c-8fd7-4698-80fd-6b98754cd837 admin admin] PATH_INFO: `/v3/projects/86d7646998114b4aa33caed11865f8a4` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:19.750834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93e7f8c-8fd7-4698-80fd-6b98754cd837 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86d7646998114b4aa33caed11865f8a4)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:19.750834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93e7f8c-8fd7-4698-80fd-6b98754cd837 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:19.792232 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f654539-b5fe-426e-be18-1cb8c9a3ac4f service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:19.797592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22b2ea-c81b-4058-b1e5-29a8084dc2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:19.798478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22b2ea-c81b-4058-b1e5-29a8084dc2ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:19.798770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22b2ea-c81b-4058-b1e5-29a8084dc2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:19.799074 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22b2ea-c81b-4058-b1e5-29a8084dc2ab admin admin] PATH_INFO: `/v3/users/f96aacb0268f4ea9b0d04d5cd0bb1d87` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:19.800415 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22b2ea-c81b-4058-b1e5-29a8084dc2ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=f96aacb0268f4ea9b0d04d5cd0bb1d87)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:19.802245 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22b2ea-c81b-4058-b1e5-29a8084dc2ab admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:19.897898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-a93e7f8c-8fd7-4698-80fd-6b98754cd837 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86d7646998114b4aa33caed11865f8a4', 'request_id': 'req-a93e7f8c-8fd7-4698-80fd-6b98754cd837'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:19.904109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-a93e7f8c-8fd7-4698-80fd-6b98754cd837 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86d7646998114b4aa33caed11865f8a4', 'request_id': 'req-a93e7f8c-8fd7-4698-80fd-6b98754cd837'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:19.910226 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-a93e7f8c-8fd7-4698-80fd-6b98754cd837 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86d7646998114b4aa33caed11865f8a4', 'request_id': 'req-a93e7f8c-8fd7-4698-80fd-6b98754cd837'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:20.018214 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-a93e7f8c-8fd7-4698-80fd-6b98754cd837 admin admin] The token cache is being invalidate because project 86d7646998114b4aa33caed11865f8a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:25:20.018590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-a93e7f8c-8fd7-4698-80fd-6b98754cd837 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:20.019353 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-a93e7f8c-8fd7-4698-80fd-6b98754cd837 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:20.021958 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1000/2008] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:25:19 2022] DELETE /identity/v3/projects/86d7646998114b4aa33caed11865f8a4 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:20.024670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e55ac-bafd-403f-81ef-415aded69776 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:20.024670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e55ac-bafd-403f-81ef-415aded69776 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:20.024670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e55ac-bafd-403f-81ef-415aded69776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:20.031779 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7f22b2ea-c81b-4058-b1e5-29a8084dc2ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f96aacb0268f4ea9b0d04d5cd0bb1d87'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:20.040080 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7f22b2ea-c81b-4058-b1e5-29a8084dc2ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f96aacb0268f4ea9b0d04d5cd0bb1d87'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:20.045807 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7f22b2ea-c81b-4058-b1e5-29a8084dc2ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f96aacb0268f4ea9b0d04d5cd0bb1d87'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:20.045807 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7f22b2ea-c81b-4058-b1e5-29a8084dc2ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f96aacb0268f4ea9b0d04d5cd0bb1d87'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:20.048585 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-854e55ac-bafd-403f-81ef-415aded69776 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:20.050466 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-854e55ac-bafd-403f-81ef-415aded69776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:20.054002 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7f22b2ea-c81b-4058-b1e5-29a8084dc2ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f96aacb0268f4ea9b0d04d5cd0bb1d87'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:20.108778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1001/2009] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:20.110219 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e55ac-bafd-403f-81ef-415aded69776 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:20.111508 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-854e55ac-bafd-403f-81ef-415aded69776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:20.123253 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e55ac-bafd-403f-81ef-415aded69776 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:20.127982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a28e8-519c-49ea-9795-3599fe98ecbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:20.128848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a28e8-519c-49ea-9795-3599fe98ecbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:20.129156 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a28e8-519c-49ea-9795-3599fe98ecbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:20.129440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a28e8-519c-49ea-9795-3599fe98ecbf admin admin] PATH_INFO: `/v3/users/431570aae7d64813a4d6899f39270846` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:20.130720 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a28e8-519c-49ea-9795-3599fe98ecbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=431570aae7d64813a4d6899f39270846)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:20.132683 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a28e8-519c-49ea-9795-3599fe98ecbf admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:20.161529 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1009/2010] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:25:19 2022] DELETE /identity/v3/users/f96aacb0268f4ea9b0d04d5cd0bb1d87 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:20.161529 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71494df-e5dd-42eb-ab04-b082b894efcb None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:20.161529 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71494df-e5dd-42eb-ab04-b082b894efcb None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:20.161529 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71494df-e5dd-42eb-ab04-b082b894efcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:20.180139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-f71494df-e5dd-42eb-ab04-b082b894efcb None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:20.181396 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-f71494df-e5dd-42eb-ab04-b082b894efcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:20.245800 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1010/2011] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:20.246670 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71494df-e5dd-42eb-ab04-b082b894efcb None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:20.248075 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-f71494df-e5dd-42eb-ab04-b082b894efcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:20.309568 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71494df-e5dd-42eb-ab04-b082b894efcb None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:20.309568 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecca40b-e2de-4e3f-b2d4-c218235cc620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:20.309845 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecca40b-e2de-4e3f-b2d4-c218235cc620 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:20.310123 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecca40b-e2de-4e3f-b2d4-c218235cc620 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:20.310399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecca40b-e2de-4e3f-b2d4-c218235cc620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:20.320306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecca40b-e2de-4e3f-b2d4-c218235cc620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:20.321752 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecca40b-e2de-4e3f-b2d4-c218235cc620 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:20.331824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1011/2012] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:25:20.333283 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7aedd-dc03-43c5-9ab3-c85f8d2003f5 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:20.333584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7aedd-dc03-43c5-9ab3-c85f8d2003f5 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:20.333872 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7aedd-dc03-43c5-9ab3-c85f8d2003f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:20.364675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-d0c7aedd-dc03-43c5-9ab3-c85f8d2003f5 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:20.365916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d0c7aedd-dc03-43c5-9ab3-c85f8d2003f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:20.378548 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1012/2013] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:20.380197 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7aedd-dc03-43c5-9ab3-c85f8d2003f5 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:20.390839 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7aedd-dc03-43c5-9ab3-c85f8d2003f5 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:20.399610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cc078-0e20-4a25-a789-4493fa28eafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:20.399610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cc078-0e20-4a25-a789-4493fa28eafa admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:20.399610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cc078-0e20-4a25-a789-4493fa28eafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:20.399610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cc078-0e20-4a25-a789-4493fa28eafa admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:20.400194 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cc078-0e20-4a25-a789-4493fa28eafa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:25:20.400194 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cc078-0e20-4a25-a789-4493fa28eafa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:20.401412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cc078-0e20-4a25-a789-4493fa28eafa admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:20.415180 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1013/2014] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:25:20.419186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cc078-0e20-4a25-a789-4493fa28eafa admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:20.430538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cc078-0e20-4a25-a789-4493fa28eafa admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:20.443903 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87fbb42-2cfe-49ea-b056-2f1129fefd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:20.443903 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fbb42-2cfe-49ea-b056-2f1129fefd88 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:20.443903 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fbb42-2cfe-49ea-b056-2f1129fefd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:20.443903 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fbb42-2cfe-49ea-b056-2f1129fefd88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:20.444386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87fbb42-2cfe-49ea-b056-2f1129fefd88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:20.444386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87fbb42-2cfe-49ea-b056-2f1129fefd88 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:20.483325 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-3e1a28e8-519c-49ea-9795-3599fe98ecbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '431570aae7d64813a4d6899f39270846'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:20.508831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-3e1a28e8-519c-49ea-9795-3599fe98ecbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '431570aae7d64813a4d6899f39270846'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:20.514277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-3e1a28e8-519c-49ea-9795-3599fe98ecbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '431570aae7d64813a4d6899f39270846'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:20.514277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-3e1a28e8-519c-49ea-9795-3599fe98ecbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '431570aae7d64813a4d6899f39270846'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:20.534987 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-3e1a28e8-519c-49ea-9795-3599fe98ecbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '431570aae7d64813a4d6899f39270846'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:20.583376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1014/2015] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:25:20 2022] POST /identity/v3/projects => generated 383 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:25:20.589458 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87fbb42-2cfe-49ea-b056-2f1129fefd88 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:20.591241 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-e87fbb42-2cfe-49ea-b056-2f1129fefd88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:20.672564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87fbb42-2cfe-49ea-b056-2f1129fefd88 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:20.672960 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1002/2016] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:25:20 2022] DELETE /identity/v3/users/431570aae7d64813a4d6899f39270846 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:20.676756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da73ec-7277-4f83-b684-d3af92ee8ca1 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:20.676756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da73ec-7277-4f83-b684-d3af92ee8ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:20.676756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da73ec-7277-4f83-b684-d3af92ee8ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:20.700217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-08da73ec-7277-4f83-b684-d3af92ee8ca1 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:20.701532 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-08da73ec-7277-4f83-b684-d3af92ee8ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:20.760935 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76431e22-3500-4692-be19-ef4925c7b774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:20.761614 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76431e22-3500-4692-be19-ef4925c7b774 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:20.761959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76431e22-3500-4692-be19-ef4925c7b774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:20.762386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76431e22-3500-4692-be19-ef4925c7b774 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:20.763193 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76431e22-3500-4692-be19-ef4925c7b774 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:20.765292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76431e22-3500-4692-be19-ef4925c7b774 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:20.771482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1003/2017] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:25:20.783416 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da73ec-7277-4f83-b684-d3af92ee8ca1 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:20.784791 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-08da73ec-7277-4f83-b684-d3af92ee8ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:20.847507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da73ec-7277-4f83-b684-d3af92ee8ca1 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:20.851466 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674c279-3ecb-4ba7-b34a-a29c00a1958b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:20.859891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c279-3ecb-4ba7-b34a-a29c00a1958b service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:20.859891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c279-3ecb-4ba7-b34a-a29c00a1958b service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:20.859891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c279-3ecb-4ba7-b34a-a29c00a1958b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:20.864112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674c279-3ecb-4ba7-b34a-a29c00a1958b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:20.865555 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674c279-3ecb-4ba7-b34a-a29c00a1958b service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:20.875470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1004/2018] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:25:20.876278 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674c279-3ecb-4ba7-b34a-a29c00a1958b service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:20.877872 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e674c279-3ecb-4ba7-b34a-a29c00a1958b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:20.887685 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1015/2019] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:25:20 2022] POST /identity/v3/users => generated 292 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:25:20.889257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76431e22-3500-4692-be19-ef4925c7b774 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:20.891595 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-76431e22-3500-4692-be19-ef4925c7b774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:20.891777 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674c279-3ecb-4ba7-b34a-a29c00a1958b service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:20.896901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c8d801-ec54-4ddf-8785-db47e41c720c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:20.897810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8d801-ec54-4ddf-8785-db47e41c720c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:20.898212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8d801-ec54-4ddf-8785-db47e41c720c admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:20.898546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8d801-ec54-4ddf-8785-db47e41c720c admin admin] PATH_INFO: `/v3/projects/6009eebfd4a84bc193cae411e2621ca2` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:20.900036 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c8d801-ec54-4ddf-8785-db47e41c720c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6009eebfd4a84bc193cae411e2621ca2)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:20.902668 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c8d801-ec54-4ddf-8785-db47e41c720c admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:20.910258 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76431e22-3500-4692-be19-ef4925c7b774 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:20.917435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ae5a0-79fb-46e7-9dfa-13290746ac65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:20.918477 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ae5a0-79fb-46e7-9dfa-13290746ac65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:20.918973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ae5a0-79fb-46e7-9dfa-13290746ac65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:20.919547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ae5a0-79fb-46e7-9dfa-13290746ac65 admin admin] PATH_INFO: `/v3/projects/cf8494b864f24d89ac729bd9fd8517c7` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:20.921428 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ae5a0-79fb-46e7-9dfa-13290746ac65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf8494b864f24d89ac729bd9fd8517c7)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:20.925565 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ae5a0-79fb-46e7-9dfa-13290746ac65 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:21.017720 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b7c8d801-ec54-4ddf-8785-db47e41c720c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6009eebfd4a84bc193cae411e2621ca2', 'request_id': 'req-b7c8d801-ec54-4ddf-8785-db47e41c720c'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:21.024085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b7c8d801-ec54-4ddf-8785-db47e41c720c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6009eebfd4a84bc193cae411e2621ca2', 'request_id': 'req-b7c8d801-ec54-4ddf-8785-db47e41c720c'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:21.029705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b7c8d801-ec54-4ddf-8785-db47e41c720c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6009eebfd4a84bc193cae411e2621ca2', 'request_id': 'req-b7c8d801-ec54-4ddf-8785-db47e41c720c'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:21.074966 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d89ae5a0-79fb-46e7-9dfa-13290746ac65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf8494b864f24d89ac729bd9fd8517c7', 'request_id': 'req-d89ae5a0-79fb-46e7-9dfa-13290746ac65'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:21.081467 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d89ae5a0-79fb-46e7-9dfa-13290746ac65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf8494b864f24d89ac729bd9fd8517c7', 'request_id': 'req-d89ae5a0-79fb-46e7-9dfa-13290746ac65'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:21.173204 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b7c8d801-ec54-4ddf-8785-db47e41c720c admin admin] The token cache is being invalidate because project 6009eebfd4a84bc193cae411e2621ca2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:25:21.173204 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b7c8d801-ec54-4ddf-8785-db47e41c720c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:21.173762 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b7c8d801-ec54-4ddf-8785-db47e41c720c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:21.175530 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1005/2020] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:25:20 2022] DELETE /identity/v3/projects/6009eebfd4a84bc193cae411e2621ca2 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:21.176541 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c8d801-ec54-4ddf-8785-db47e41c720c admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:21.177967 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b7c8d801-ec54-4ddf-8785-db47e41c720c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:21.245640 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c8d801-ec54-4ddf-8785-db47e41c720c admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:21.301173 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23f494-e14b-41a5-aa05-9e8ce4ab9d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:21.301850 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23f494-e14b-41a5-aa05-9e8ce4ab9d06 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:21.302135 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23f494-e14b-41a5-aa05-9e8ce4ab9d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:21.302414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23f494-e14b-41a5-aa05-9e8ce4ab9d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:21.303184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa23f494-e14b-41a5-aa05-9e8ce4ab9d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:21.304781 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa23f494-e14b-41a5-aa05-9e8ce4ab9d06 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:21.308811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d89ae5a0-79fb-46e7-9dfa-13290746ac65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf8494b864f24d89ac729bd9fd8517c7', 'request_id': 'req-d89ae5a0-79fb-46e7-9dfa-13290746ac65'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:21.315737 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1006/2021] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:25:21.317736 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd33bf-ef37-48b1-9796-0a8f7f740f19 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:21.317736 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd33bf-ef37-48b1-9796-0a8f7f740f19 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:21.317736 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd33bf-ef37-48b1-9796-0a8f7f740f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:21.323888 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d89ae5a0-79fb-46e7-9dfa-13290746ac65 admin admin] The token cache is being invalidate because project cf8494b864f24d89ac729bd9fd8517c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:25:21.324180 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d89ae5a0-79fb-46e7-9dfa-13290746ac65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:21.324834 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d89ae5a0-79fb-46e7-9dfa-13290746ac65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:21.327234 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1016/2022] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:25:20 2022] DELETE /identity/v3/projects/cf8494b864f24d89ac729bd9fd8517c7 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:21.328538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ae5a0-79fb-46e7-9dfa-13290746ac65 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:21.329824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d89ae5a0-79fb-46e7-9dfa-13290746ac65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:21.339705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-93fd33bf-ef37-48b1-9796-0a8f7f740f19 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:21.340905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-93fd33bf-ef37-48b1-9796-0a8f7f740f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:21.344276 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ae5a0-79fb-46e7-9dfa-13290746ac65 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:21.349005 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1007/2023] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:21.355350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1070d8f-df78-496c-8b79-4ce0b1af4bf6 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:21.355350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1070d8f-df78-496c-8b79-4ce0b1af4bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:21.355350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1070d8f-df78-496c-8b79-4ce0b1af4bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:21.355674 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e850d-b888-4e91-a682-7979e4ece15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:21.355674 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e850d-b888-4e91-a682-7979e4ece15f admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:21.355674 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e850d-b888-4e91-a682-7979e4ece15f admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:21.355674 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e850d-b888-4e91-a682-7979e4ece15f admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:21.356026 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e850d-b888-4e91-a682-7979e4ece15f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:21.356026 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e850d-b888-4e91-a682-7979e4ece15f admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:21.370267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1017/2024] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:25:21.370267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4658e-5b2a-452d-8e05-6b29541f2671 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:21.370267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4658e-5b2a-452d-8e05-6b29541f2671 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:21.370267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4658e-5b2a-452d-8e05-6b29541f2671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:21.372977 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-f1070d8f-df78-496c-8b79-4ce0b1af4bf6 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:21.374168 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-f1070d8f-df78-496c-8b79-4ce0b1af4bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:21.382646 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1008/2025] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:21.389332 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1070d8f-df78-496c-8b79-4ce0b1af4bf6 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:21.393253 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1070d8f-df78-496c-8b79-4ce0b1af4bf6 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:21.394236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-21e4658e-5b2a-452d-8e05-6b29541f2671 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:21.395543 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-21e4658e-5b2a-452d-8e05-6b29541f2671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:21.399642 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769200b-09bd-4342-9580-24196b6cef21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:21.400656 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769200b-09bd-4342-9580-24196b6cef21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:21.400861 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769200b-09bd-4342-9580-24196b6cef21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:21.401267 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769200b-09bd-4342-9580-24196b6cef21 admin admin] PATH_INFO: `/v3/projects/a3e90badfd504467be6e283bfc4ab539/users/13339ca30d6947809a51ad761bd9a7ca/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:21.404529 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1018/2026] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:21.405861 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44c49d-75ce-4348-8300-082e26e70e3c None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:21.406297 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44c49d-75ce-4348-8300-082e26e70e3c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:21.406676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44c49d-75ce-4348-8300-082e26e70e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:21.424617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769200b-09bd-4342-9580-24196b6cef21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3e90badfd504467be6e283bfc4ab539, user_id=13339ca30d6947809a51ad761bd9a7ca, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:21.428046 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769200b-09bd-4342-9580-24196b6cef21 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:21.441070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-8b44c49d-75ce-4348-8300-082e26e70e3c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:21.442277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-8b44c49d-75ce-4348-8300-082e26e70e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:21.451097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1019/2027] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:21.452003 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44c49d-75ce-4348-8300-082e26e70e3c None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:21.453277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-8b44c49d-75ce-4348-8300-082e26e70e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:21.470100 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44c49d-75ce-4348-8300-082e26e70e3c None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:21.474224 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346929f-274d-4b04-ad74-49ca962f9f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:21.475131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346929f-274d-4b04-ad74-49ca962f9f9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:21.475413 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346929f-274d-4b04-ad74-49ca962f9f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:21.475748 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346929f-274d-4b04-ad74-49ca962f9f9f admin admin] PATH_INFO: `/v3/projects/d8faa464772d41a9b71e36e2c0c5d57f` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:21.476987 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346929f-274d-4b04-ad74-49ca962f9f9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8faa464772d41a9b71e36e2c0c5d57f)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:21.479004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346929f-274d-4b04-ad74-49ca962f9f9f admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:21.534721 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1009/2028] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:25:21 2022] PUT /identity/v3/projects/a3e90badfd504467be6e283bfc4ab539/users/13339ca30d6947809a51ad761bd9a7ca/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:21.534721 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769200b-09bd-4342-9580-24196b6cef21 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:21.589558 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:21.632429 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769200b-09bd-4342-9580-24196b6cef21 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:21.650835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-8346929f-274d-4b04-ad74-49ca962f9f9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8faa464772d41a9b71e36e2c0c5d57f', 'request_id': 'req-8346929f-274d-4b04-ad74-49ca962f9f9f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:21.659310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-8346929f-274d-4b04-ad74-49ca962f9f9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8faa464772d41a9b71e36e2c0c5d57f', 'request_id': 'req-8346929f-274d-4b04-ad74-49ca962f9f9f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:21.713915 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c39049-ef99-4169-842d-5a7422fdb32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:21.714597 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c39049-ef99-4169-842d-5a7422fdb32f admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:21.714894 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c39049-ef99-4169-842d-5a7422fdb32f admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:21.715209 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c39049-ef99-4169-842d-5a7422fdb32f admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:21.716030 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c39049-ef99-4169-842d-5a7422fdb32f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:25:21.716344 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c39049-ef99-4169-842d-5a7422fdb32f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:21.718125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c39049-ef99-4169-842d-5a7422fdb32f admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:21.731319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1010/2029] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:25:21.732286 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c39049-ef99-4169-842d-5a7422fdb32f admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:21.745361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c39049-ef99-4169-842d-5a7422fdb32f admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:21.755547 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5326e-2955-4382-b013-19334abfadf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:21.755863 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5326e-2955-4382-b013-19334abfadf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:21.756044 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-8346929f-274d-4b04-ad74-49ca962f9f9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8faa464772d41a9b71e36e2c0c5d57f', 'request_id': 'req-8346929f-274d-4b04-ad74-49ca962f9f9f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:21.756295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5326e-2955-4382-b013-19334abfadf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:21.757507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5326e-2955-4382-b013-19334abfadf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:21.757507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5326e-2955-4382-b013-19334abfadf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:25:21.757896 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5326e-2955-4382-b013-19334abfadf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:21.759913 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5326e-2955-4382-b013-19334abfadf2 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:21.773829 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1011/2030] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:25:21.775723 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ef557-354c-4856-a37f-719c16fd5cc1 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:21.776108 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ef557-354c-4856-a37f-719c16fd5cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:21.776445 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ef557-354c-4856-a37f-719c16fd5cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:21.782541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-8346929f-274d-4b04-ad74-49ca962f9f9f admin admin] The token cache is being invalidate because project d8faa464772d41a9b71e36e2c0c5d57f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:25:21.782870 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-8346929f-274d-4b04-ad74-49ca962f9f9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:21.783873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-8346929f-274d-4b04-ad74-49ca962f9f9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:21.787753 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1020/2031] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:25:21 2022] DELETE /identity/v3/projects/d8faa464772d41a9b71e36e2c0c5d57f => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:21.792238 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346929f-274d-4b04-ad74-49ca962f9f9f admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:21.799412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-8346929f-274d-4b04-ad74-49ca962f9f9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:21.815383 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-fa7ef557-354c-4856-a37f-719c16fd5cc1 None None] MFA Rules not processed for user `13339ca30d6947809a51ad761bd9a7ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:21.817169 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-fa7ef557-354c-4856-a37f-719c16fd5cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:21.818800 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346929f-274d-4b04-ad74-49ca962f9f9f admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:21.824001 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ae543-3e51-4c89-97fa-f868ca1fd2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:21.825148 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ae543-3e51-4c89-97fa-f868ca1fd2eb admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:21.825148 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ae543-3e51-4c89-97fa-f868ca1fd2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:21.825347 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ae543-3e51-4c89-97fa-f868ca1fd2eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:21.825933 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ae543-3e51-4c89-97fa-f868ca1fd2eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:21.828202 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ae543-3e51-4c89-97fa-f868ca1fd2eb admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:21.888816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1012/2032] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:21.889647 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ef557-354c-4856-a37f-719c16fd5cc1 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:21.892056 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-fa7ef557-354c-4856-a37f-719c16fd5cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:21.909825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ef557-354c-4856-a37f-719c16fd5cc1 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:21.914590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641b7fa-375a-41f6-9501-038a205bfe6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:21.915290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641b7fa-375a-41f6-9501-038a205bfe6d admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:21.915580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641b7fa-375a-41f6-9501-038a205bfe6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:21.915855 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641b7fa-375a-41f6-9501-038a205bfe6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:21.916701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641b7fa-375a-41f6-9501-038a205bfe6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:21.918675 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641b7fa-375a-41f6-9501-038a205bfe6d admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:21.935039 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1021/2033] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:25:21 2022] POST /identity/v3/projects => generated 381 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:25:21.942454 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5311e4b-6c2e-43ee-8413-9b982f783657 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:21.942454 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5311e4b-6c2e-43ee-8413-9b982f783657 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:21.942454 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5311e4b-6c2e-43ee-8413-9b982f783657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:21.955169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-d5311e4b-6c2e-43ee-8413-9b982f783657 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:21.956410 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d5311e4b-6c2e-43ee-8413-9b982f783657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:22.014940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1022/2034] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:22.016362 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5311e4b-6c2e-43ee-8413-9b982f783657 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:22.017623 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d5311e4b-6c2e-43ee-8413-9b982f783657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:22.029277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5311e4b-6c2e-43ee-8413-9b982f783657 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:22.035035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba90fc-a34b-4075-b866-53acb4d166fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:22.035690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba90fc-a34b-4075-b866-53acb4d166fe admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:22.035975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba90fc-a34b-4075-b866-53acb4d166fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:22.036253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba90fc-a34b-4075-b866-53acb4d166fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:22.036949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba90fc-a34b-4075-b866-53acb4d166fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:22.038869 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba90fc-a34b-4075-b866-53acb4d166fe admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:22.050240 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1013/2035] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:25:21 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:25:22.051583 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641b7fa-375a-41f6-9501-038a205bfe6d admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:22.120243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641b7fa-375a-41f6-9501-038a205bfe6d admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:22.124910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64301de-0530-4c8a-99bc-5ac8678a0b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:22.125582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64301de-0530-4c8a-99bc-5ac8678a0b80 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:22.125868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64301de-0530-4c8a-99bc-5ac8678a0b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:22.126174 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64301de-0530-4c8a-99bc-5ac8678a0b80 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:22.126870 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64301de-0530-4c8a-99bc-5ac8678a0b80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:22.128842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64301de-0530-4c8a-99bc-5ac8678a0b80 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:22.179143 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1023/2036] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:25:22 2022] POST /identity/v3/projects => generated 383 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:25:22.180740 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbec183-197e-40fe-9c3c-2714523f49ac None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:22.181188 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbec183-197e-40fe-9c3c-2714523f49ac None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:22.181476 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbec183-197e-40fe-9c3c-2714523f49ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:22.200114 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-2fbec183-197e-40fe-9c3c-2714523f49ac None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:22.201239 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-2fbec183-197e-40fe-9c3c-2714523f49ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:22.242720 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1014/2037] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:25:22 2022] POST /identity/v3/users => generated 291 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:25:22.243854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64301de-0530-4c8a-99bc-5ac8678a0b80 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:22.254938 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64301de-0530-4c8a-99bc-5ac8678a0b80 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:22.266561 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1024/2038] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:22.267556 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbec183-197e-40fe-9c3c-2714523f49ac None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:22.277411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbec183-197e-40fe-9c3c-2714523f49ac None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:22.282886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864470ce-3a5f-405e-834a-8306c68e4b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:22.283590 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864470ce-3a5f-405e-834a-8306c68e4b36 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:22.283878 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864470ce-3a5f-405e-834a-8306c68e4b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:22.284163 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864470ce-3a5f-405e-834a-8306c68e4b36 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:22.284870 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864470ce-3a5f-405e-834a-8306c68e4b36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:22.287094 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864470ce-3a5f-405e-834a-8306c68e4b36 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:22.320192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb3363-de2b-45b0-9612-eaf2e0c1005d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:22.320888 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb3363-de2b-45b0-9612-eaf2e0c1005d admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:22.321727 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb3363-de2b-45b0-9612-eaf2e0c1005d admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:22.322029 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb3363-de2b-45b0-9612-eaf2e0c1005d admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:22.322784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb3363-de2b-45b0-9612-eaf2e0c1005d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:22.324764 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb3363-de2b-45b0-9612-eaf2e0c1005d admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:22.375518 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1025/2039] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:25:22 2022] POST /identity/v3/users => generated 298 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:25:22.375518 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864470ce-3a5f-405e-834a-8306c68e4b36 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:22.385207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864470ce-3a5f-405e-834a-8306c68e4b36 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:22.391190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc71c2e-7e05-46b3-b679-abb8edbaf75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:22.392013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc71c2e-7e05-46b3-b679-abb8edbaf75d admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:22.392581 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc71c2e-7e05-46b3-b679-abb8edbaf75d admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:22.392967 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc71c2e-7e05-46b3-b679-abb8edbaf75d admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:22.393707 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc71c2e-7e05-46b3-b679-abb8edbaf75d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:22.395392 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc71c2e-7e05-46b3-b679-abb8edbaf75d admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:22.409746 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1015/2040] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:25:22 2022] POST /identity/v3/users => generated 288 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:25:22.409940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1026/2041] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:25:22.410694 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb3363-de2b-45b0-9612-eaf2e0c1005d admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:22.411993 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc71c2e-7e05-46b3-b679-abb8edbaf75d admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:22.422074 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb3363-de2b-45b0-9612-eaf2e0c1005d admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:22.424979 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc71c2e-7e05-46b3-b679-abb8edbaf75d admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:22.427918 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150777b-f594-4f66-876f-7a3c6aaa999d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:22.428663 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150777b-f594-4f66-876f-7a3c6aaa999d admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:22.428961 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150777b-f594-4f66-876f-7a3c6aaa999d admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:22.429249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150777b-f594-4f66-876f-7a3c6aaa999d admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:22.429991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b150777b-f594-4f66-876f-7a3c6aaa999d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:25:22.430306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b150777b-f594-4f66-876f-7a3c6aaa999d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:22.431136 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f61126-c9cf-42b6-9f46-23b0fd98da41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:22.431586 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f61126-c9cf-42b6-9f46-23b0fd98da41 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:22.431958 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b150777b-f594-4f66-876f-7a3c6aaa999d admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:22.432072 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f61126-c9cf-42b6-9f46-23b0fd98da41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:22.432818 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f61126-c9cf-42b6-9f46-23b0fd98da41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:22.433949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f61126-c9cf-42b6-9f46-23b0fd98da41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:22.435685 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f61126-c9cf-42b6-9f46-23b0fd98da41 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:22.447113 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1016/2042] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:25:22.447999 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150777b-f594-4f66-876f-7a3c6aaa999d admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:22.451143 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1027/2043] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:25:22.454734 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f61126-c9cf-42b6-9f46-23b0fd98da41 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:22.459321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150777b-f594-4f66-876f-7a3c6aaa999d admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:22.464885 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38794a94-1f39-47f0-a480-fa07fe60f966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:22.465560 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38794a94-1f39-47f0-a480-fa07fe60f966 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:22.465905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38794a94-1f39-47f0-a480-fa07fe60f966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:22.466185 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38794a94-1f39-47f0-a480-fa07fe60f966 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:22.467189 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38794a94-1f39-47f0-a480-fa07fe60f966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:22.467550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f61126-c9cf-42b6-9f46-23b0fd98da41 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:22.470679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38794a94-1f39-47f0-a480-fa07fe60f966 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:22.472649 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f3162-6a1f-4445-a5bb-5f555a8f83bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:22.473725 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f3162-6a1f-4445-a5bb-5f555a8f83bf admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:22.474153 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f3162-6a1f-4445-a5bb-5f555a8f83bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:22.474448 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f3162-6a1f-4445-a5bb-5f555a8f83bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:22.475223 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f3162-6a1f-4445-a5bb-5f555a8f83bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:22.476845 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f3162-6a1f-4445-a5bb-5f555a8f83bf admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:22.479659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1017/2044] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:25:22.480450 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38794a94-1f39-47f0-a480-fa07fe60f966 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:22.488051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1028/2045] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:25:22.489049 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f3162-6a1f-4445-a5bb-5f555a8f83bf admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:22.494733 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38794a94-1f39-47f0-a480-fa07fe60f966 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:22.500349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061197f-5801-4b20-b53a-47e20a120e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:22.501019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061197f-5801-4b20-b53a-47e20a120e35 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:22.501305 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061197f-5801-4b20-b53a-47e20a120e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:22.501771 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f3162-6a1f-4445-a5bb-5f555a8f83bf admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:22.501880 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061197f-5801-4b20-b53a-47e20a120e35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:22.502453 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061197f-5801-4b20-b53a-47e20a120e35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:22.504458 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061197f-5801-4b20-b53a-47e20a120e35 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:22.506177 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29dbf91-9763-4006-b772-3043928508ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:22.507088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dbf91-9763-4006-b772-3043928508ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:22.507374 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dbf91-9763-4006-b772-3043928508ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:22.507677 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dbf91-9763-4006-b772-3043928508ef admin admin] PATH_INFO: `/v3/projects/aaac0ab583b344a189dfd1799386bd16/users/aad94cf7c44541e4bd6c9f0d9f8e38e7/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:22.525154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29dbf91-9763-4006-b772-3043928508ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaac0ab583b344a189dfd1799386bd16, user_id=aad94cf7c44541e4bd6c9f0d9f8e38e7, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:22.528017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29dbf91-9763-4006-b772-3043928508ef admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:22.617782 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1029/2046] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:25:22 2022] PUT /identity/v3/projects/aaac0ab583b344a189dfd1799386bd16/users/aad94cf7c44541e4bd6c9f0d9f8e38e7/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:22.618865 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29dbf91-9763-4006-b772-3043928508ef admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:22.622159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1018/2047] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:25:22 2022] POST /identity/v3/projects => generated 389 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:25:22.622159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061197f-5801-4b20-b53a-47e20a120e35 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:22.691597 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:22.694562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061197f-5801-4b20-b53a-47e20a120e35 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:22.700441 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c94b5-57db-4eb8-98aa-0e937a0db238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:22.704873 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c94b5-57db-4eb8-98aa-0e937a0db238 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:22.705609 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c94b5-57db-4eb8-98aa-0e937a0db238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:22.705609 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c94b5-57db-4eb8-98aa-0e937a0db238 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:22.707094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c94b5-57db-4eb8-98aa-0e937a0db238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:22.708751 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c94b5-57db-4eb8-98aa-0e937a0db238 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:22.729429 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1019/2048] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:25:22.730818 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c94b5-57db-4eb8-98aa-0e937a0db238 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:22.745342 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c94b5-57db-4eb8-98aa-0e937a0db238 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:22.751465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7c305-fe21-4a90-babc-f4bdb76859e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:22.752442 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7c305-fe21-4a90-babc-f4bdb76859e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:22.759713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7c305-fe21-4a90-babc-f4bdb76859e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:22.759713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7c305-fe21-4a90-babc-f4bdb76859e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:22.759713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a7c305-fe21-4a90-babc-f4bdb76859e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:22.759713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a7c305-fe21-4a90-babc-f4bdb76859e9 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:22.760879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29dbf91-9763-4006-b772-3043928508ef admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:22.770139 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1020/2049] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:25:22.771491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7c305-fe21-4a90-babc-f4bdb76859e9 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:22.775840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3980ca-d3f3-4969-9fcc-8d5ce3728828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:22.784598 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7c305-fe21-4a90-babc-f4bdb76859e9 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:22.788368 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3980ca-d3f3-4969-9fcc-8d5ce3728828 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:22.788368 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3980ca-d3f3-4969-9fcc-8d5ce3728828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:22.788368 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3980ca-d3f3-4969-9fcc-8d5ce3728828 admin admin] PATH_INFO: `/v3/projects/8fd425794c144fdba7d43b3fe22297da/users/b913dbe0099e4f21b4c2c1fa4c2bd390/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:22.791362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845ea21-cf4e-49c3-9810-67d3c7a525e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:22.792897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845ea21-cf4e-49c3-9810-67d3c7a525e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:22.793536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845ea21-cf4e-49c3-9810-67d3c7a525e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:22.794020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845ea21-cf4e-49c3-9810-67d3c7a525e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:22.794950 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845ea21-cf4e-49c3-9810-67d3c7a525e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:22.798068 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845ea21-cf4e-49c3-9810-67d3c7a525e3 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:22.820269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3980ca-d3f3-4969-9fcc-8d5ce3728828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd425794c144fdba7d43b3fe22297da, user_id=b913dbe0099e4f21b4c2c1fa4c2bd390, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:22.824755 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3980ca-d3f3-4969-9fcc-8d5ce3728828 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:22.972545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1021/2050] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:25:22 2022] POST /identity/v3/users => generated 295 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:25:22.973550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845ea21-cf4e-49c3-9810-67d3c7a525e3 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:22.983693 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845ea21-cf4e-49c3-9810-67d3c7a525e3 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:22.989152 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027acbb-4d4a-4016-945b-436aadd8b59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:22.990020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027acbb-4d4a-4016-945b-436aadd8b59d admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:22.990310 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027acbb-4d4a-4016-945b-436aadd8b59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:22.990594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027acbb-4d4a-4016-945b-436aadd8b59d admin admin] PATH_INFO: `/v3/projects/f47f6750e93744caa2562e14bedaaa29/users/363d9d9e5ce145dc996309af1a60bb2a/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:23.013346 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027acbb-4d4a-4016-945b-436aadd8b59d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f47f6750e93744caa2562e14bedaaa29, user_id=363d9d9e5ce145dc996309af1a60bb2a, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:23.018368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027acbb-4d4a-4016-945b-436aadd8b59d admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:23.018499 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1030/2051] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:25:22 2022] PUT /identity/v3/projects/8fd425794c144fdba7d43b3fe22297da/users/b913dbe0099e4f21b4c2c1fa4c2bd390/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:23.027103 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3980ca-d3f3-4969-9fcc-8d5ce3728828 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:23.098651 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3980ca-d3f3-4969-9fcc-8d5ce3728828 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:23.104776 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c8960-1d0b-4804-b21f-dbb0365a2314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:23.105774 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c8960-1d0b-4804-b21f-dbb0365a2314 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:23.106134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c8960-1d0b-4804-b21f-dbb0365a2314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:23.106482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c8960-1d0b-4804-b21f-dbb0365a2314 admin admin] PATH_INFO: `/v3/projects/aaac0ab583b344a189dfd1799386bd16/users/aad94cf7c44541e4bd6c9f0d9f8e38e7/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:23.108962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c8960-1d0b-4804-b21f-dbb0365a2314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaac0ab583b344a189dfd1799386bd16, user_id=aad94cf7c44541e4bd6c9f0d9f8e38e7, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:23.120034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1022/2052] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:25:22 2022] PUT /identity/v3/projects/f47f6750e93744caa2562e14bedaaa29/users/363d9d9e5ce145dc996309af1a60bb2a/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:23.120034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027acbb-4d4a-4016-945b-436aadd8b59d admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:23.120240 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c8960-1d0b-4804-b21f-dbb0365a2314 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:23.125321 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1031/2053] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:25:23 2022] PUT /identity/v3/projects/aaac0ab583b344a189dfd1799386bd16/users/aad94cf7c44541e4bd6c9f0d9f8e38e7/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:23.126746 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522f7d7-4598-421a-870b-696402c33738 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:23.127177 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522f7d7-4598-421a-870b-696402c33738 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:23.127562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522f7d7-4598-421a-870b-696402c33738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:23.167565 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-d522f7d7-4598-421a-870b-696402c33738 None None] MFA Rules not processed for user `b913dbe0099e4f21b4c2c1fa4c2bd390`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:23.171657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d522f7d7-4598-421a-870b-696402c33738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:23.232827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027acbb-4d4a-4016-945b-436aadd8b59d admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:23.301359 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1032/2054] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:23.302763 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04b451-78c4-4327-81a2-c4372a890a56 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:23.303103 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04b451-78c4-4327-81a2-c4372a890a56 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:23.303390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04b451-78c4-4327-81a2-c4372a890a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:23.340207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-1c04b451-78c4-4327-81a2-c4372a890a56 None None] MFA Rules not processed for user `363d9d9e5ce145dc996309af1a60bb2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:23.341795 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-1c04b451-78c4-4327-81a2-c4372a890a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:23.350968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e3ac9-c882-4071-a269-e3759fb1d88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:23.351677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e3ac9-c882-4071-a269-e3759fb1d88e admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:23.351966 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e3ac9-c882-4071-a269-e3759fb1d88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:23.352246 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e3ac9-c882-4071-a269-e3759fb1d88e admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:23.352987 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e3ac9-c882-4071-a269-e3759fb1d88e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:23.354726 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e3ac9-c882-4071-a269-e3759fb1d88e admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:23.375044 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1023/2055] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:25:23.378820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b88f4-ec13-471f-9e6b-5ca16e203c15 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:23.379777 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b88f4-ec13-471f-9e6b-5ca16e203c15 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:23.380075 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b88f4-ec13-471f-9e6b-5ca16e203c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:23.425077 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1033/2056] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:23.426171 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04b451-78c4-4327-81a2-c4372a890a56 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:23.441499 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:23.460507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-716b88f4-ec13-471f-9e6b-5ca16e203c15 None None] MFA Rules not processed for user `aad94cf7c44541e4bd6c9f0d9f8e38e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:23.462503 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-716b88f4-ec13-471f-9e6b-5ca16e203c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:23.465385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04b451-78c4-4327-81a2-c4372a890a56 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:23.477204 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfde861-7c6c-48e6-82dc-858ea8a00719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:23.478031 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfde861-7c6c-48e6-82dc-858ea8a00719 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:23.478327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfde861-7c6c-48e6-82dc-858ea8a00719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:23.479131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfde861-7c6c-48e6-82dc-858ea8a00719 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:23.480073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfde861-7c6c-48e6-82dc-858ea8a00719 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:23.482252 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfde861-7c6c-48e6-82dc-858ea8a00719 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:23.589703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1034/2057] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:25:23 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:25:23.594722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfde861-7c6c-48e6-82dc-858ea8a00719 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:23.614094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1024/2058] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:23.614094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b88f4-ec13-471f-9e6b-5ca16e203c15 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:23.614094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-716b88f4-ec13-471f-9e6b-5ca16e203c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:23.681938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfde861-7c6c-48e6-82dc-858ea8a00719 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:23.686105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b88f4-ec13-471f-9e6b-5ca16e203c15 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:23.687574 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea654bec-89db-4775-ade4-15bab13ed4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:23.688270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea654bec-89db-4775-ade4-15bab13ed4fe admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:23.688564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea654bec-89db-4775-ade4-15bab13ed4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:23.688848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea654bec-89db-4775-ade4-15bab13ed4fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:23.689603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea654bec-89db-4775-ade4-15bab13ed4fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:23.691276 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea654bec-89db-4775-ade4-15bab13ed4fe admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:23.702454 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1035/2059] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:25:23.707245 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea654bec-89db-4775-ade4-15bab13ed4fe admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:23.719041 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea654bec-89db-4775-ade4-15bab13ed4fe admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:23.724000 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7d3e6-be28-4eff-8682-f7e97b7eb442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:23.724832 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7d3e6-be28-4eff-8682-f7e97b7eb442 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:23.725183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7d3e6-be28-4eff-8682-f7e97b7eb442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:23.725475 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7d3e6-be28-4eff-8682-f7e97b7eb442 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:23.726211 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7d3e6-be28-4eff-8682-f7e97b7eb442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:23.728204 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7d3e6-be28-4eff-8682-f7e97b7eb442 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:23.744711 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7387dd-9fcb-47eb-a673-9757560dba3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:23.745484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7387dd-9fcb-47eb-a673-9757560dba3b service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:23.745780 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7387dd-9fcb-47eb-a673-9757560dba3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:23.746070 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7387dd-9fcb-47eb-a673-9757560dba3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:23.816659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7387dd-9fcb-47eb-a673-9757560dba3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:23.818135 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7387dd-9fcb-47eb-a673-9757560dba3b service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:23.829395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1025/2060] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:25:23.830538 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7387dd-9fcb-47eb-a673-9757560dba3b service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:23.832217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-3b7387dd-9fcb-47eb-a673-9757560dba3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:23.845250 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7387dd-9fcb-47eb-a673-9757560dba3b service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:23.851319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35644850-f2bb-4842-8be6-a85bbf569924 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:23.858640 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35644850-f2bb-4842-8be6-a85bbf569924 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:23.858640 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35644850-f2bb-4842-8be6-a85bbf569924 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:23.858640 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35644850-f2bb-4842-8be6-a85bbf569924 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:23.859042 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1036/2061] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:25:23 2022] POST /identity/v3/users => generated 296 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:25:23.859042 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7d3e6-be28-4eff-8682-f7e97b7eb442 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:23.873827 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7d3e6-be28-4eff-8682-f7e97b7eb442 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:23.880547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3d284-b9b2-4488-8f9d-e6be59198514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:23.881282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3d284-b9b2-4488-8f9d-e6be59198514 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:23.882010 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3d284-b9b2-4488-8f9d-e6be59198514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:23.882297 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3d284-b9b2-4488-8f9d-e6be59198514 admin admin] PATH_INFO: `/v3/projects/8360cbb90a9745408f7423ecd57f17ae/users/f1aa8f446c6e4dcba32e5cd4ff0c141d/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:23.912575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3d284-b9b2-4488-8f9d-e6be59198514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8360cbb90a9745408f7423ecd57f17ae, user_id=f1aa8f446c6e4dcba32e5cd4ff0c141d, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:23.916155 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3d284-b9b2-4488-8f9d-e6be59198514 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:23.978093 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35644850-f2bb-4842-8be6-a85bbf569924 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:23.980804 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35644850-f2bb-4842-8be6-a85bbf569924 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:23.998775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1026/2062] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:25:24.000729 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35644850-f2bb-4842-8be6-a85bbf569924 service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:24.007914 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:24.020960 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1037/2063] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:25:23 2022] PUT /identity/v3/projects/8360cbb90a9745408f7423ecd57f17ae/users/f1aa8f446c6e4dcba32e5cd4ff0c141d/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:24.032058 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b6f68-f2f8-4dd4-9469-e23eb5c340cb None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:24.032355 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b6f68-f2f8-4dd4-9469-e23eb5c340cb None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:24.032642 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b6f68-f2f8-4dd4-9469-e23eb5c340cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:24.034614 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35644850-f2bb-4842-8be6-a85bbf569924 service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:24.069051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-3a4b6f68-f2f8-4dd4-9469-e23eb5c340cb None None] MFA Rules not processed for user `f1aa8f446c6e4dcba32e5cd4ff0c141d`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:24.070800 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-3a4b6f68-f2f8-4dd4-9469-e23eb5c340cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:24.120568 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7fcf3-9087-4044-a682-e757268d3aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:24.121473 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7fcf3-9087-4044-a682-e757268d3aef admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:24.121891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7fcf3-9087-4044-a682-e757268d3aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:24.122650 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7fcf3-9087-4044-a682-e757268d3aef admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:24.123044 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7fcf3-9087-4044-a682-e757268d3aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:24.125307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7fcf3-9087-4044-a682-e757268d3aef admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:24.140623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1027/2064] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:25:24.144868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7fcf3-9087-4044-a682-e757268d3aef admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:24.169171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7fcf3-9087-4044-a682-e757268d3aef admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:24.173892 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1038/2065] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:24.178374 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955b126-216b-4974-aad8-a94892e2e4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:24.179353 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955b126-216b-4974-aad8-a94892e2e4c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:24.179643 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955b126-216b-4974-aad8-a94892e2e4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:24.179930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955b126-216b-4974-aad8-a94892e2e4c2 admin admin] PATH_INFO: `/v3/projects/f5ce21da98784ea7a9d854cfb385c777/users/2d147bb333e141aba8d9ff68ae31ef5f/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:24.197648 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955b126-216b-4974-aad8-a94892e2e4c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5ce21da98784ea7a9d854cfb385c777, user_id=2d147bb333e141aba8d9ff68ae31ef5f, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:24.200867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955b126-216b-4974-aad8-a94892e2e4c2 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:24.318103 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1028/2066] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:25:24 2022] PUT /identity/v3/projects/f5ce21da98784ea7a9d854cfb385c777/users/2d147bb333e141aba8d9ff68ae31ef5f/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:24.324362 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b6f68-f2f8-4dd4-9469-e23eb5c340cb None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:24.336331 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b6f68-f2f8-4dd4-9469-e23eb5c340cb None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:24.407990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f977bbe-8f28-462b-b4d2-30fe3c92036c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:24.408745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f977bbe-8f28-462b-b4d2-30fe3c92036c admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:24.409041 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f977bbe-8f28-462b-b4d2-30fe3c92036c admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:24.409328 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f977bbe-8f28-462b-b4d2-30fe3c92036c admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:24.410141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f977bbe-8f28-462b-b4d2-30fe3c92036c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:24.411815 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f977bbe-8f28-462b-b4d2-30fe3c92036c admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:24.423659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1039/2067] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:25:24.427803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955b126-216b-4974-aad8-a94892e2e4c2 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:24.439478 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955b126-216b-4974-aad8-a94892e2e4c2 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:24.444592 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f829ae-95af-41a9-ab0d-4b18746e3064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:24.445501 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f829ae-95af-41a9-ab0d-4b18746e3064 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:24.445801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f829ae-95af-41a9-ab0d-4b18746e3064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:24.446083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f829ae-95af-41a9-ab0d-4b18746e3064 admin admin] PATH_INFO: `/v3/projects/f5ce21da98784ea7a9d854cfb385c777/users/2d147bb333e141aba8d9ff68ae31ef5f/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:24.448249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f829ae-95af-41a9-ab0d-4b18746e3064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5ce21da98784ea7a9d854cfb385c777, user_id=2d147bb333e141aba8d9ff68ae31ef5f, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:24.451007 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f829ae-95af-41a9-ab0d-4b18746e3064 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:24.462053 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1029/2068] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:25:24 2022] PUT /identity/v3/projects/f5ce21da98784ea7a9d854cfb385c777/users/2d147bb333e141aba8d9ff68ae31ef5f/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:25:24.472809 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab6fd2-d068-416b-9cfe-78c75de6fe26 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:24.472809 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab6fd2-d068-416b-9cfe-78c75de6fe26 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:24.473441 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab6fd2-d068-416b-9cfe-78c75de6fe26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:24.518162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-71ab6fd2-d068-416b-9cfe-78c75de6fe26 None None] MFA Rules not processed for user `2d147bb333e141aba8d9ff68ae31ef5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:24.519688 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-71ab6fd2-d068-416b-9cfe-78c75de6fe26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:24.595317 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1040/2069] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:24.754049 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:24.816178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f829ae-95af-41a9-ab0d-4b18746e3064 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:24.891584 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f829ae-95af-41a9-ab0d-4b18746e3064 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:24.898122 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a118f73-5379-45a4-a9ef-b69f1e56bc12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:24.898983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a118f73-5379-45a4-a9ef-b69f1e56bc12 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:24.899326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a118f73-5379-45a4-a9ef-b69f1e56bc12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:24.899699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a118f73-5379-45a4-a9ef-b69f1e56bc12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:24.978124 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a118f73-5379-45a4-a9ef-b69f1e56bc12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:24.980322 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a118f73-5379-45a4-a9ef-b69f1e56bc12 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:24.993272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1030/2070] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:25:25.023612 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab6fd2-d068-416b-9cfe-78c75de6fe26 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:25.035397 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab6fd2-d068-416b-9cfe-78c75de6fe26 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:25.039149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0539e-4650-479e-86ba-7ee553c84e46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:25.039806 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0539e-4650-479e-86ba-7ee553c84e46 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:25.040094 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0539e-4650-479e-86ba-7ee553c84e46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:25.040375 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0539e-4650-479e-86ba-7ee553c84e46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:25.053453 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0539e-4650-479e-86ba-7ee553c84e46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:25.055298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0539e-4650-479e-86ba-7ee553c84e46 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:25.068407 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1041/2071] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:25:25.176457 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:25.752426 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a118f73-5379-45a4-a9ef-b69f1e56bc12 service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:25.761294 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a118f73-5379-45a4-a9ef-b69f1e56bc12 service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:25.765854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d26cbd-43a7-4214-85ee-c5dea23ae0f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:25.766520 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d26cbd-43a7-4214-85ee-c5dea23ae0f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:25.766808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d26cbd-43a7-4214-85ee-c5dea23ae0f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:25.767130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d26cbd-43a7-4214-85ee-c5dea23ae0f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:25.837563 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d26cbd-43a7-4214-85ee-c5dea23ae0f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:25.840394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d26cbd-43a7-4214-85ee-c5dea23ae0f7 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:25.865200 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1031/2072] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:25:26.202627 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0539e-4650-479e-86ba-7ee553c84e46 service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:26.205110 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5ef0539e-4650-479e-86ba-7ee553c84e46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:26.270111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0539e-4650-479e-86ba-7ee553c84e46 service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:26.274687 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74cf76-4b05-4118-8711-f2327bfeaea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:26.275727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74cf76-4b05-4118-8711-f2327bfeaea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:26.276014 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74cf76-4b05-4118-8711-f2327bfeaea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:26.276290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74cf76-4b05-4118-8711-f2327bfeaea0 admin admin] PATH_INFO: `/v3/users/6e7f2d909d45471fb0b97ddb724cb195` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:26.277538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74cf76-4b05-4118-8711-f2327bfeaea0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e7f2d909d45471fb0b97ddb724cb195)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:26.279478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74cf76-4b05-4118-8711-f2327bfeaea0 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:26.523110 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:26.544029 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0d74cf76-4b05-4118-8711-f2327bfeaea0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e7f2d909d45471fb0b97ddb724cb195'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:26.556845 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0d74cf76-4b05-4118-8711-f2327bfeaea0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e7f2d909d45471fb0b97ddb724cb195'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:26.557734 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0d74cf76-4b05-4118-8711-f2327bfeaea0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e7f2d909d45471fb0b97ddb724cb195'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:26.558359 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0d74cf76-4b05-4118-8711-f2327bfeaea0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e7f2d909d45471fb0b97ddb724cb195'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:26.572287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0d74cf76-4b05-4118-8711-f2327bfeaea0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e7f2d909d45471fb0b97ddb724cb195'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:26.720407 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1042/2073] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:25:26 2022] DELETE /identity/v3/users/6e7f2d909d45471fb0b97ddb724cb195 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:27.004522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:27.042998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d26cbd-43a7-4214-85ee-c5dea23ae0f7 service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:27.055962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d26cbd-43a7-4214-85ee-c5dea23ae0f7 service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:27.120009 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad010f20-9f65-4b78-b982-e1e75339c296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:27.120879 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad010f20-9f65-4b78-b982-e1e75339c296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:27.121166 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad010f20-9f65-4b78-b982-e1e75339c296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:27.121446 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad010f20-9f65-4b78-b982-e1e75339c296 admin admin] PATH_INFO: `/v3/projects/0160aecb23854482b26d188beec1021d` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:27.122649 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad010f20-9f65-4b78-b982-e1e75339c296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0160aecb23854482b26d188beec1021d)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:27.125363 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad010f20-9f65-4b78-b982-e1e75339c296 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:27.247229 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ad010f20-9f65-4b78-b982-e1e75339c296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0160aecb23854482b26d188beec1021d', 'request_id': 'req-ad010f20-9f65-4b78-b982-e1e75339c296'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:27.254084 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ad010f20-9f65-4b78-b982-e1e75339c296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0160aecb23854482b26d188beec1021d', 'request_id': 'req-ad010f20-9f65-4b78-b982-e1e75339c296'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:27.259673 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ad010f20-9f65-4b78-b982-e1e75339c296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0160aecb23854482b26d188beec1021d', 'request_id': 'req-ad010f20-9f65-4b78-b982-e1e75339c296'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:27.383250 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ad010f20-9f65-4b78-b982-e1e75339c296 admin admin] The token cache is being invalidate because project 0160aecb23854482b26d188beec1021d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:25:27.383555 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ad010f20-9f65-4b78-b982-e1e75339c296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:27.385298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ad010f20-9f65-4b78-b982-e1e75339c296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:27.387518 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1032/2074] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:25:27 2022] DELETE /identity/v3/projects/0160aecb23854482b26d188beec1021d => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:27.395036 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d258a4e-32c2-4714-b934-3c4a2081e1bd None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:27.395547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d258a4e-32c2-4714-b934-3c4a2081e1bd None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:27.395641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d258a4e-32c2-4714-b934-3c4a2081e1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:27.420281 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-4d258a4e-32c2-4714-b934-3c4a2081e1bd None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:27.422399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-4d258a4e-32c2-4714-b934-3c4a2081e1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:27.519178 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1043/2075] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:27.527094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017017c-8ccd-4fdc-92d6-46ab83b79bc1 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:27.527636 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017017c-8ccd-4fdc-92d6-46ab83b79bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:27.527636 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017017c-8ccd-4fdc-92d6-46ab83b79bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:27.548304 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-2017017c-8ccd-4fdc-92d6-46ab83b79bc1 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:27.549213 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-2017017c-8ccd-4fdc-92d6-46ab83b79bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:27.557542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1033/2076] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:27.562053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d258a4e-32c2-4714-b934-3c4a2081e1bd None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:27.572422 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d258a4e-32c2-4714-b934-3c4a2081e1bd None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:27.577343 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a0281-7f82-4d0e-9b3b-af084cbbb55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:27.578037 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a0281-7f82-4d0e-9b3b-af084cbbb55a admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:27.579038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a0281-7f82-4d0e-9b3b-af084cbbb55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:27.579038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a0281-7f82-4d0e-9b3b-af084cbbb55a admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:27.579364 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a0281-7f82-4d0e-9b3b-af084cbbb55a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:25:27.579669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a0281-7f82-4d0e-9b3b-af084cbbb55a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:27.581486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a0281-7f82-4d0e-9b3b-af084cbbb55a admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:27.594261 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1044/2077] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:25:27.598747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017017c-8ccd-4fdc-92d6-46ab83b79bc1 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:27.609938 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017017c-8ccd-4fdc-92d6-46ab83b79bc1 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:27.614685 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca82f4a-133f-419e-b0ce-73e309825632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:27.615388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca82f4a-133f-419e-b0ce-73e309825632 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:27.615679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca82f4a-133f-419e-b0ce-73e309825632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:27.615959 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca82f4a-133f-419e-b0ce-73e309825632 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:27.616688 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca82f4a-133f-419e-b0ce-73e309825632 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:27.618589 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca82f4a-133f-419e-b0ce-73e309825632 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:27.785005 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1034/2078] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:25:27 2022] POST /identity/v3/projects => generated 383 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:25:27.789277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a0281-7f82-4d0e-9b3b-af084cbbb55a admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:27.800025 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a0281-7f82-4d0e-9b3b-af084cbbb55a admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:27.860484 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f43189-f8b5-4b37-b652-ebdd326c5887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:27.861174 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f43189-f8b5-4b37-b652-ebdd326c5887 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:27.861479 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f43189-f8b5-4b37-b652-ebdd326c5887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:27.861758 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f43189-f8b5-4b37-b652-ebdd326c5887 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:27.862456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f43189-f8b5-4b37-b652-ebdd326c5887 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:27.864509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f43189-f8b5-4b37-b652-ebdd326c5887 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:27.972782 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1045/2079] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:25:27 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:25:27.976931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca82f4a-133f-419e-b0ce-73e309825632 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:27.987814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca82f4a-133f-419e-b0ce-73e309825632 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:27.992544 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5e5ac-ae9c-481c-a8c3-b83b8443d5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:27.993324 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5e5ac-ae9c-481c-a8c3-b83b8443d5c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:27.993615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5e5ac-ae9c-481c-a8c3-b83b8443d5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:27.993897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5e5ac-ae9c-481c-a8c3-b83b8443d5c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:27.994660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa5e5ac-ae9c-481c-a8c3-b83b8443d5c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:27.996297 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa5e5ac-ae9c-481c-a8c3-b83b8443d5c2 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:28.006786 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1035/2080] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:25:28.010974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f43189-f8b5-4b37-b652-ebdd326c5887 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:28.021718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f43189-f8b5-4b37-b652-ebdd326c5887 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:28.026366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f56efe-7954-4e04-9480-0f665ea01010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:28.027058 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f56efe-7954-4e04-9480-0f665ea01010 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:28.027349 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f56efe-7954-4e04-9480-0f665ea01010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:28.027632 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f56efe-7954-4e04-9480-0f665ea01010 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:28.028365 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f56efe-7954-4e04-9480-0f665ea01010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:28.029978 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f56efe-7954-4e04-9480-0f665ea01010 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:28.040478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1046/2081] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:25:28.044634 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5e5ac-ae9c-481c-a8c3-b83b8443d5c2 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:28.054536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5e5ac-ae9c-481c-a8c3-b83b8443d5c2 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:28.059227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c4c3f-c278-464e-a15f-3dd563ab8e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:28.060087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c4c3f-c278-464e-a15f-3dd563ab8e9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:28.060371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c4c3f-c278-464e-a15f-3dd563ab8e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:28.060648 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c4c3f-c278-464e-a15f-3dd563ab8e9e admin admin] PATH_INFO: `/v3/projects/8d0795f7c61d497c9b25c7e17ad08744/users/ad5452341ccc40bc94b38369dae4f2b8/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:28.075713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c4c3f-c278-464e-a15f-3dd563ab8e9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d0795f7c61d497c9b25c7e17ad08744, user_id=ad5452341ccc40bc94b38369dae4f2b8, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:28.079713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c4c3f-c278-464e-a15f-3dd563ab8e9e admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:28.264543 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1036/2082] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:25:28 2022] PUT /identity/v3/projects/8d0795f7c61d497c9b25c7e17ad08744/users/ad5452341ccc40bc94b38369dae4f2b8/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:28.270167 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577febb-91c5-4f69-8394-3a9f5e1d5977 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:28.270467 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577febb-91c5-4f69-8394-3a9f5e1d5977 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:28.270754 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577febb-91c5-4f69-8394-3a9f5e1d5977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:28.291157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-6577febb-91c5-4f69-8394-3a9f5e1d5977 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:28.292674 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-6577febb-91c5-4f69-8394-3a9f5e1d5977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:28.356831 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1047/2083] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:28.367280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c4c3f-c278-464e-a15f-3dd563ab8e9e admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:28.369050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-af4c4c3f-c278-464e-a15f-3dd563ab8e9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:28.426704 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c4c3f-c278-464e-a15f-3dd563ab8e9e admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:28.430550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3410ed-20c0-465e-bf9d-2d72a5b8f668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:28.431231 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3410ed-20c0-465e-bf9d-2d72a5b8f668 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:28.431521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3410ed-20c0-465e-bf9d-2d72a5b8f668 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:28.431800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3410ed-20c0-465e-bf9d-2d72a5b8f668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:28.441701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3410ed-20c0-465e-bf9d-2d72a5b8f668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:28.443193 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3410ed-20c0-465e-bf9d-2d72a5b8f668 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:28.453170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1037/2084] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:25:28.520369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:28.994971 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:29.424374 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:29.915072 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:30.000902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:31.143415 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:31.582475 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:31.740452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:33.021660 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:33.362225 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:34.527208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:35.394266 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227f809-9d35-4f8b-aa30-f55afb78688c None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:35.394559 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227f809-9d35-4f8b-aa30-f55afb78688c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:35.394838 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227f809-9d35-4f8b-aa30-f55afb78688c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:35.422797 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-c227f809-9d35-4f8b-aa30-f55afb78688c None None] MFA Rules not processed for user `ad5452341ccc40bc94b38369dae4f2b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:35.424029 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-c227f809-9d35-4f8b-aa30-f55afb78688c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:35.528901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1048/2085] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:35.545433 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3410ed-20c0-465e-bf9d-2d72a5b8f668 service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:35.550256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-0b3410ed-20c0-465e-bf9d-2d72a5b8f668 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:35.561061 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:35.587945 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:35.650040 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3410ed-20c0-465e-bf9d-2d72a5b8f668 service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:35.657054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779338c4-0868-4b6d-9388-50837613379d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:35.657401 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779338c4-0868-4b6d-9388-50837613379d service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:35.657686 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779338c4-0868-4b6d-9388-50837613379d service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:35.658131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779338c4-0868-4b6d-9388-50837613379d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:35.668307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779338c4-0868-4b6d-9388-50837613379d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:35.670133 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779338c4-0868-4b6d-9388-50837613379d service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:35.691249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1038/2086] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:25:37.116429 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:38.887435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227f809-9d35-4f8b-aa30-f55afb78688c None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:38.889409 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-c227f809-9d35-4f8b-aa30-f55afb78688c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:38.965942 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227f809-9d35-4f8b-aa30-f55afb78688c None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:38.969777 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40482f4a-5706-47d1-945a-a7bda8ce99bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:38.971205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40482f4a-5706-47d1-945a-a7bda8ce99bf service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:38.971816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40482f4a-5706-47d1-945a-a7bda8ce99bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:38.972174 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40482f4a-5706-47d1-945a-a7bda8ce99bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:38.974173 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-40482f4a-5706-47d1-945a-a7bda8ce99bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:38.997754 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdfa9a-22f5-4f5d-9bf4-934d4bba3154 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:38.997754 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdfa9a-22f5-4f5d-9bf4-934d4bba3154 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:38.997754 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdfa9a-22f5-4f5d-9bf4-934d4bba3154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:39.017089 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-47cdfa9a-22f5-4f5d-9bf4-934d4bba3154 None None] MFA Rules not processed for user `ad5452341ccc40bc94b38369dae4f2b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:39.018322 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-47cdfa9a-22f5-4f5d-9bf4-934d4bba3154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:39.027641 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1039/2087] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:25:39.045005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40482f4a-5706-47d1-945a-a7bda8ce99bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:39.046582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40482f4a-5706-47d1-945a-a7bda8ce99bf service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:39.057445 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1049/2088] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:25:49.011448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=29 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:49.361145 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=96 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:54.414333 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdfa9a-22f5-4f5d-9bf4-934d4bba3154 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:54.416100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-47cdfa9a-22f5-4f5d-9bf4-934d4bba3154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:54.435192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdfa9a-22f5-4f5d-9bf4-934d4bba3154 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:54.444823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df327e47-579c-423a-9ee6-e703c371cc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:54.445724 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df327e47-579c-423a-9ee6-e703c371cc6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:54.446044 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df327e47-579c-423a-9ee6-e703c371cc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:54.446324 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df327e47-579c-423a-9ee6-e703c371cc6b admin admin] PATH_INFO: `/v3/users/b913dbe0099e4f21b4c2c1fa4c2bd390` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:54.447569 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df327e47-579c-423a-9ee6-e703c371cc6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b913dbe0099e4f21b4c2c1fa4c2bd390)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:54.449556 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df327e47-579c-423a-9ee6-e703c371cc6b admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:54.705701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-df327e47-579c-423a-9ee6-e703c371cc6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b913dbe0099e4f21b4c2c1fa4c2bd390'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:54.712986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-df327e47-579c-423a-9ee6-e703c371cc6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b913dbe0099e4f21b4c2c1fa4c2bd390'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:54.713669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-df327e47-579c-423a-9ee6-e703c371cc6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b913dbe0099e4f21b4c2c1fa4c2bd390'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:54.714280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-df327e47-579c-423a-9ee6-e703c371cc6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b913dbe0099e4f21b4c2c1fa4c2bd390'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:54.726956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-df327e47-579c-423a-9ee6-e703c371cc6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b913dbe0099e4f21b4c2c1fa4c2bd390'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:54.830444 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1040/2089] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:25:54 2022] DELETE /identity/v3/users/b913dbe0099e4f21b4c2c1fa4c2bd390 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:54.835534 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a1c39-1cba-4292-ac01-ef8a1bc38fd4 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:54.836036 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a1c39-1cba-4292-ac01-ef8a1bc38fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:54.836388 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a1c39-1cba-4292-ac01-ef8a1bc38fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:54.862276 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-001a1c39-1cba-4292-ac01-ef8a1bc38fd4 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:54.863369 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-001a1c39-1cba-4292-ac01-ef8a1bc38fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:54.968701 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1050/2090] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:55.004842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df327e47-579c-423a-9ee6-e703c371cc6b admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:55.007344 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-df327e47-579c-423a-9ee6-e703c371cc6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:55.071988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a1c39-1cba-4292-ac01-ef8a1bc38fd4 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:55.074473 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-001a1c39-1cba-4292-ac01-ef8a1bc38fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:55.102754 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a1c39-1cba-4292-ac01-ef8a1bc38fd4 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:55.123381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ec8ee-d209-4393-bc15-5829d0107595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:55.125094 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ec8ee-d209-4393-bc15-5829d0107595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:55.125389 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ec8ee-d209-4393-bc15-5829d0107595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:55.130362 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ec8ee-d209-4393-bc15-5829d0107595 admin admin] PATH_INFO: `/v3/users/13339ca30d6947809a51ad761bd9a7ca` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:55.130362 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ec8ee-d209-4393-bc15-5829d0107595 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13339ca30d6947809a51ad761bd9a7ca)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:55.130940 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df327e47-579c-423a-9ee6-e703c371cc6b admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:55.131063 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ec8ee-d209-4393-bc15-5829d0107595 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:55.135142 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2858030-9743-4508-bfa4-ed1f0eb6ed9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:55.135959 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2858030-9743-4508-bfa4-ed1f0eb6ed9b service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:55.136403 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2858030-9743-4508-bfa4-ed1f0eb6ed9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:55.136834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2858030-9743-4508-bfa4-ed1f0eb6ed9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:55.153255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2858030-9743-4508-bfa4-ed1f0eb6ed9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:55.154788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2858030-9743-4508-bfa4-ed1f0eb6ed9b service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:55.177170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1041/2091] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:25:55.296094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2858030-9743-4508-bfa4-ed1f0eb6ed9b service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:55.297853 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e2858030-9743-4508-bfa4-ed1f0eb6ed9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:55.316332 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2858030-9743-4508-bfa4-ed1f0eb6ed9b service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:55.321733 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981d09c-d625-461c-9439-7e54e771e02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:55.322906 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981d09c-d625-461c-9439-7e54e771e02d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:55.323421 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981d09c-d625-461c-9439-7e54e771e02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:55.324343 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981d09c-d625-461c-9439-7e54e771e02d admin admin] PATH_INFO: `/v3/users/f1aa8f446c6e4dcba32e5cd4ff0c141d` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:55.325900 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981d09c-d625-461c-9439-7e54e771e02d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1aa8f446c6e4dcba32e5cd4ff0c141d)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:55.328785 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981d09c-d625-461c-9439-7e54e771e02d admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:55.369017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1f6ec8ee-d209-4393-bc15-5829d0107595 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13339ca30d6947809a51ad761bd9a7ca'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:55.378448 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1f6ec8ee-d209-4393-bc15-5829d0107595 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13339ca30d6947809a51ad761bd9a7ca'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:55.379226 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1f6ec8ee-d209-4393-bc15-5829d0107595 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13339ca30d6947809a51ad761bd9a7ca'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:55.380296 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1f6ec8ee-d209-4393-bc15-5829d0107595 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13339ca30d6947809a51ad761bd9a7ca'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:55.395154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1f6ec8ee-d209-4393-bc15-5829d0107595 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13339ca30d6947809a51ad761bd9a7ca'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:55.516243 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1051/2092] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:25:55 2022] DELETE /identity/v3/users/13339ca30d6947809a51ad761bd9a7ca => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:55.521239 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63e429-f8ec-4db0-b2bf-3680f996b348 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:55.521547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63e429-f8ec-4db0-b2bf-3680f996b348 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:55.521814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63e429-f8ec-4db0-b2bf-3680f996b348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:55.544093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-0f63e429-f8ec-4db0-b2bf-3680f996b348 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:55.546350 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-0f63e429-f8ec-4db0-b2bf-3680f996b348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:55.614303 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1052/2093] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:55.623740 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63e429-f8ec-4db0-b2bf-3680f996b348 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:55.625175 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-0f63e429-f8ec-4db0-b2bf-3680f996b348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:55.640150 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:55.652200 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-5981d09c-d625-461c-9439-7e54e771e02d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1aa8f446c6e4dcba32e5cd4ff0c141d'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:55.663743 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-5981d09c-d625-461c-9439-7e54e771e02d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1aa8f446c6e4dcba32e5cd4ff0c141d'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:55.665535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-5981d09c-d625-461c-9439-7e54e771e02d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1aa8f446c6e4dcba32e5cd4ff0c141d'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:55.666363 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-5981d09c-d625-461c-9439-7e54e771e02d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1aa8f446c6e4dcba32e5cd4ff0c141d'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:55.680928 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:55.682720 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-5981d09c-d625-461c-9439-7e54e771e02d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1aa8f446c6e4dcba32e5cd4ff0c141d'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:55.685022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63e429-f8ec-4db0-b2bf-3680f996b348 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:55.691491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ec94b-99f2-46f7-9668-853753ee0c44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:55.692225 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ec94b-99f2-46f7-9668-853753ee0c44 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:55.692518 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ec94b-99f2-46f7-9668-853753ee0c44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:55.692795 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ec94b-99f2-46f7-9668-853753ee0c44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:55.694592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ed1ec94b-99f2-46f7-9668-853753ee0c44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:55.707454 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ec94b-99f2-46f7-9668-853753ee0c44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:55.708899 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ec94b-99f2-46f7-9668-853753ee0c44 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:55.720013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1053/2094] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:25:55.862834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1042/2095] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:25:55 2022] DELETE /identity/v3/users/f1aa8f446c6e4dcba32e5cd4ff0c141d => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:25:55.867481 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5cd64-4e6d-4994-8b43-51c2c448edfd None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:55.867962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5cd64-4e6d-4994-8b43-51c2c448edfd None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:55.868271 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5cd64-4e6d-4994-8b43-51c2c448edfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:55.890736 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-78e5cd64-4e6d-4994-8b43-51c2c448edfd None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:55.891975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-78e5cd64-4e6d-4994-8b43-51c2c448edfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:55.927703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981d09c-d625-461c-9439-7e54e771e02d admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:55.929558 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-5981d09c-d625-461c-9439-7e54e771e02d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:55.981657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1054/2096] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:25:55.998202 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5cd64-4e6d-4994-8b43-51c2c448edfd None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:55.998202 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-78e5cd64-4e6d-4994-8b43-51c2c448edfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:56.013347 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981d09c-d625-461c-9439-7e54e771e02d admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:56.024860 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464344a6-5821-4779-a7c7-08a3ddee7d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:56.026777 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464344a6-5821-4779-a7c7-08a3ddee7d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:56.026777 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464344a6-5821-4779-a7c7-08a3ddee7d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:56.026777 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464344a6-5821-4779-a7c7-08a3ddee7d05 admin admin] PATH_INFO: `/v3/projects/8fd425794c144fdba7d43b3fe22297da` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:56.027808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464344a6-5821-4779-a7c7-08a3ddee7d05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fd425794c144fdba7d43b3fe22297da)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:56.029764 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464344a6-5821-4779-a7c7-08a3ddee7d05 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:56.050675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5cd64-4e6d-4994-8b43-51c2c448edfd None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:56.055379 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6e650-2b09-465f-92a1-5a15bb90df2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:56.056253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6e650-2b09-465f-92a1-5a15bb90df2e service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:56.056532 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6e650-2b09-465f-92a1-5a15bb90df2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:56.057017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6e650-2b09-465f-92a1-5a15bb90df2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:56.068199 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6e650-2b09-465f-92a1-5a15bb90df2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:56.069636 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6e650-2b09-465f-92a1-5a15bb90df2e service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:56.079952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1055/2097] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:25:56.176016 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-464344a6-5821-4779-a7c7-08a3ddee7d05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fd425794c144fdba7d43b3fe22297da', 'request_id': 'req-464344a6-5821-4779-a7c7-08a3ddee7d05'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:56.183058 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-464344a6-5821-4779-a7c7-08a3ddee7d05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fd425794c144fdba7d43b3fe22297da', 'request_id': 'req-464344a6-5821-4779-a7c7-08a3ddee7d05'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:56.202616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-464344a6-5821-4779-a7c7-08a3ddee7d05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fd425794c144fdba7d43b3fe22297da', 'request_id': 'req-464344a6-5821-4779-a7c7-08a3ddee7d05'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:56.359207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-464344a6-5821-4779-a7c7-08a3ddee7d05 admin admin] The token cache is being invalidate because project 8fd425794c144fdba7d43b3fe22297da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:25:56.359823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-464344a6-5821-4779-a7c7-08a3ddee7d05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:56.361174 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-464344a6-5821-4779-a7c7-08a3ddee7d05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:56.361553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6e650-2b09-465f-92a1-5a15bb90df2e service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:56.363204 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a4d6e650-2b09-465f-92a1-5a15bb90df2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:56.363343 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1043/2098] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:25:55 2022] DELETE /identity/v3/projects/8fd425794c144fdba7d43b3fe22297da => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:25:56.368011 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464344a6-5821-4779-a7c7-08a3ddee7d05 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:56.369615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-464344a6-5821-4779-a7c7-08a3ddee7d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:56.380992 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6e650-2b09-465f-92a1-5a15bb90df2e service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:56.498461 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa229b-c937-4c85-8089-c45f3ce91c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:56.500126 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa229b-c937-4c85-8089-c45f3ce91c05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:56.500431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa229b-c937-4c85-8089-c45f3ce91c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:56.500719 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa229b-c937-4c85-8089-c45f3ce91c05 admin admin] PATH_INFO: `/v3/projects/a3e90badfd504467be6e283bfc4ab539` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:56.502254 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa229b-c937-4c85-8089-c45f3ce91c05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3e90badfd504467be6e283bfc4ab539)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:56.504261 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa229b-c937-4c85-8089-c45f3ce91c05 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:56.507618 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464344a6-5821-4779-a7c7-08a3ddee7d05 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:56.513187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8c7c3-13d8-4ba8-90c1-45f7f7c87bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:56.514094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8c7c3-13d8-4ba8-90c1-45f7f7c87bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:56.514982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8c7c3-13d8-4ba8-90c1-45f7f7c87bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:56.515275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8c7c3-13d8-4ba8-90c1-45f7f7c87bd8 admin admin] PATH_INFO: `/v3/users/2d147bb333e141aba8d9ff68ae31ef5f` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:56.516526 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de8c7c3-13d8-4ba8-90c1-45f7f7c87bd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d147bb333e141aba8d9ff68ae31ef5f)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:56.518626 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de8c7c3-13d8-4ba8-90c1-45f7f7c87bd8 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:56.603215 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-36aa229b-c937-4c85-8089-c45f3ce91c05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3e90badfd504467be6e283bfc4ab539', 'request_id': 'req-36aa229b-c937-4c85-8089-c45f3ce91c05'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:56.610019 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-36aa229b-c937-4c85-8089-c45f3ce91c05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3e90badfd504467be6e283bfc4ab539', 'request_id': 'req-36aa229b-c937-4c85-8089-c45f3ce91c05'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:56.697650 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:56.698350 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-36aa229b-c937-4c85-8089-c45f3ce91c05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3e90badfd504467be6e283bfc4ab539', 'request_id': 'req-36aa229b-c937-4c85-8089-c45f3ce91c05'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:56.711239 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-36aa229b-c937-4c85-8089-c45f3ce91c05 admin admin] The token cache is being invalidate because project a3e90badfd504467be6e283bfc4ab539 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:25:56.711544 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-36aa229b-c937-4c85-8089-c45f3ce91c05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:56.712569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-36aa229b-c937-4c85-8089-c45f3ce91c05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:56.714186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1056/2099] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:25:56 2022] DELETE /identity/v3/projects/a3e90badfd504467be6e283bfc4ab539 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:25:56.721273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa229b-c937-4c85-8089-c45f3ce91c05 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:56.722953 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-36aa229b-c937-4c85-8089-c45f3ce91c05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:56.783500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-0de8c7c3-13d8-4ba8-90c1-45f7f7c87bd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d147bb333e141aba8d9ff68ae31ef5f'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:56.788672 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa229b-c937-4c85-8089-c45f3ce91c05 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:56.791714 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-0de8c7c3-13d8-4ba8-90c1-45f7f7c87bd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d147bb333e141aba8d9ff68ae31ef5f'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:56.792602 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-0de8c7c3-13d8-4ba8-90c1-45f7f7c87bd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d147bb333e141aba8d9ff68ae31ef5f'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:56.793502 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-0de8c7c3-13d8-4ba8-90c1-45f7f7c87bd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d147bb333e141aba8d9ff68ae31ef5f'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:56.795760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467786a4-6eff-48cf-b7ad-a81e2a0c96b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:56.796523 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:56.796648 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467786a4-6eff-48cf-b7ad-a81e2a0c96b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:56.796902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467786a4-6eff-48cf-b7ad-a81e2a0c96b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:56.797301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467786a4-6eff-48cf-b7ad-a81e2a0c96b4 admin admin] PATH_INFO: `/v3/projects/8360cbb90a9745408f7423ecd57f17ae` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:56.798494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467786a4-6eff-48cf-b7ad-a81e2a0c96b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8360cbb90a9745408f7423ecd57f17ae)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:56.800443 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467786a4-6eff-48cf-b7ad-a81e2a0c96b4 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:56.814937 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-0de8c7c3-13d8-4ba8-90c1-45f7f7c87bd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d147bb333e141aba8d9ff68ae31ef5f'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:56.902870 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1044/2100] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:25:56 2022] DELETE /identity/v3/users/2d147bb333e141aba8d9ff68ae31ef5f => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:25:56.904030 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8c7c3-13d8-4ba8-90c1-45f7f7c87bd8 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:56.905570 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-0de8c7c3-13d8-4ba8-90c1-45f7f7c87bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:56.966308 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-467786a4-6eff-48cf-b7ad-a81e2a0c96b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8360cbb90a9745408f7423ecd57f17ae', 'request_id': 'req-467786a4-6eff-48cf-b7ad-a81e2a0c96b4'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:56.973061 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-467786a4-6eff-48cf-b7ad-a81e2a0c96b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8360cbb90a9745408f7423ecd57f17ae', 'request_id': 'req-467786a4-6eff-48cf-b7ad-a81e2a0c96b4'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:56.998264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8c7c3-13d8-4ba8-90c1-45f7f7c87bd8 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:57.057481 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cecdd2-3b41-4140-b0fd-ae0462c46334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:57.058365 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cecdd2-3b41-4140-b0fd-ae0462c46334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:57.058656 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cecdd2-3b41-4140-b0fd-ae0462c46334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:57.058963 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cecdd2-3b41-4140-b0fd-ae0462c46334 admin admin] PATH_INFO: `/v3/users/aad94cf7c44541e4bd6c9f0d9f8e38e7` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:57.060280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cecdd2-3b41-4140-b0fd-ae0462c46334 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aad94cf7c44541e4bd6c9f0d9f8e38e7)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:57.062776 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cecdd2-3b41-4140-b0fd-ae0462c46334 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:57.079083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-467786a4-6eff-48cf-b7ad-a81e2a0c96b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8360cbb90a9745408f7423ecd57f17ae', 'request_id': 'req-467786a4-6eff-48cf-b7ad-a81e2a0c96b4'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:57.090207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-467786a4-6eff-48cf-b7ad-a81e2a0c96b4 admin admin] The token cache is being invalidate because project 8360cbb90a9745408f7423ecd57f17ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:25:57.090667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-467786a4-6eff-48cf-b7ad-a81e2a0c96b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:57.091382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-467786a4-6eff-48cf-b7ad-a81e2a0c96b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:57.093862 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1057/2101] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:25:56 2022] DELETE /identity/v3/projects/8360cbb90a9745408f7423ecd57f17ae => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:57.344787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-36cecdd2-3b41-4140-b0fd-ae0462c46334 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aad94cf7c44541e4bd6c9f0d9f8e38e7'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:57.351963 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-36cecdd2-3b41-4140-b0fd-ae0462c46334 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aad94cf7c44541e4bd6c9f0d9f8e38e7'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:57.352659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-36cecdd2-3b41-4140-b0fd-ae0462c46334 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aad94cf7c44541e4bd6c9f0d9f8e38e7'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:57.353301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-36cecdd2-3b41-4140-b0fd-ae0462c46334 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aad94cf7c44541e4bd6c9f0d9f8e38e7'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:57.366162 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-36cecdd2-3b41-4140-b0fd-ae0462c46334 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aad94cf7c44541e4bd6c9f0d9f8e38e7'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:57.508854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1045/2102] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:25:56 2022] DELETE /identity/v3/users/aad94cf7c44541e4bd6c9f0d9f8e38e7 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:57.517215 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467786a4-6eff-48cf-b7ad-a81e2a0c96b4 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:57.518747 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-467786a4-6eff-48cf-b7ad-a81e2a0c96b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:57.534053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467786a4-6eff-48cf-b7ad-a81e2a0c96b4 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:57.559447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:57.601190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4df61-d57e-4c95-81af-da28a5c0041f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:57.602184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4df61-d57e-4c95-81af-da28a5c0041f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:57.602472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4df61-d57e-4c95-81af-da28a5c0041f admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:57.602750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4df61-d57e-4c95-81af-da28a5c0041f admin admin] PATH_INFO: `/v3/projects/f5ce21da98784ea7a9d854cfb385c777` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:57.604003 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4df61-d57e-4c95-81af-da28a5c0041f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5ce21da98784ea7a9d854cfb385c777)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:57.605921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4df61-d57e-4c95-81af-da28a5c0041f admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:57.767006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d7a4df61-d57e-4c95-81af-da28a5c0041f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5ce21da98784ea7a9d854cfb385c777', 'request_id': 'req-d7a4df61-d57e-4c95-81af-da28a5c0041f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:57.773426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d7a4df61-d57e-4c95-81af-da28a5c0041f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5ce21da98784ea7a9d854cfb385c777', 'request_id': 'req-d7a4df61-d57e-4c95-81af-da28a5c0041f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:57.828844 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:57.866548 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d7a4df61-d57e-4c95-81af-da28a5c0041f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5ce21da98784ea7a9d854cfb385c777', 'request_id': 'req-d7a4df61-d57e-4c95-81af-da28a5c0041f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:57.878181 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d7a4df61-d57e-4c95-81af-da28a5c0041f admin admin] The token cache is being invalidate because project f5ce21da98784ea7a9d854cfb385c777 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:25:57.878469 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d7a4df61-d57e-4c95-81af-da28a5c0041f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:57.879136 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d7a4df61-d57e-4c95-81af-da28a5c0041f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:57.880703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1058/2103] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:25:57 2022] DELETE /identity/v3/projects/f5ce21da98784ea7a9d854cfb385c777 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:57.885581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c421df-cb36-4461-a35e-9ccc2619f3a7 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:57.885879 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c421df-cb36-4461-a35e-9ccc2619f3a7 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:57.886163 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c421df-cb36-4461-a35e-9ccc2619f3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:57.904736 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-a3c421df-cb36-4461-a35e-9ccc2619f3a7 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:57.905942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-a3c421df-cb36-4461-a35e-9ccc2619f3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:57.948388 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4df61-d57e-4c95-81af-da28a5c0041f admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:57.949918 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d7a4df61-d57e-4c95-81af-da28a5c0041f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:57.964155 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1046/2104] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:25:57.972916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248ecc1-fc6d-4d96-8c7f-f1f8d8677cfe None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:57.973214 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248ecc1-fc6d-4d96-8c7f-f1f8d8677cfe None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:57.973500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248ecc1-fc6d-4d96-8c7f-f1f8d8677cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:58.001613 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-a248ecc1-fc6d-4d96-8c7f-f1f8d8677cfe None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:58.002838 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-a248ecc1-fc6d-4d96-8c7f-f1f8d8677cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:58.011792 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1047/2105] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:25:58.015576 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248ecc1-fc6d-4d96-8c7f-f1f8d8677cfe None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:58.024681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248ecc1-fc6d-4d96-8c7f-f1f8d8677cfe None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:58.029506 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1c2be-7ebc-42d6-b5bc-a25a98c87836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:58.030244 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1c2be-7ebc-42d6-b5bc-a25a98c87836 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:58.030533 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1c2be-7ebc-42d6-b5bc-a25a98c87836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:58.030817 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1c2be-7ebc-42d6-b5bc-a25a98c87836 admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:58.031592 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1c2be-7ebc-42d6-b5bc-a25a98c87836 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:25:58.031905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1c2be-7ebc-42d6-b5bc-a25a98c87836 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:58.033656 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1c2be-7ebc-42d6-b5bc-a25a98c87836 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:58.047797 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1048/2106] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:25:58.051756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1c2be-7ebc-42d6-b5bc-a25a98c87836 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:58.062267 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1c2be-7ebc-42d6-b5bc-a25a98c87836 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:58.070659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4856f8-4e6f-4704-a3ae-5527e793bdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:58.071401 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4856f8-4e6f-4704-a3ae-5527e793bdd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:58.071691 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4856f8-4e6f-4704-a3ae-5527e793bdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:58.071973 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4856f8-4e6f-4704-a3ae-5527e793bdd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:58.072675 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4856f8-4e6f-4704-a3ae-5527e793bdd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:58.072797 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4df61-d57e-4c95-81af-da28a5c0041f admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:58.074625 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4856f8-4e6f-4704-a3ae-5527e793bdd3 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:58.078098 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a01743-8a5c-42ea-9d9f-0d9e8b2009de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:58.079097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a01743-8a5c-42ea-9d9f-0d9e8b2009de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:58.079395 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a01743-8a5c-42ea-9d9f-0d9e8b2009de admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:58.079683 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a01743-8a5c-42ea-9d9f-0d9e8b2009de admin admin] PATH_INFO: `/v3/projects/aaac0ab583b344a189dfd1799386bd16` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:58.080917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a01743-8a5c-42ea-9d9f-0d9e8b2009de admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaac0ab583b344a189dfd1799386bd16)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:58.082837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a01743-8a5c-42ea-9d9f-0d9e8b2009de admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:58.167375 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1049/2107] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:25:58 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:25:58.171166 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4856f8-4e6f-4704-a3ae-5527e793bdd3 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:58.223200 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-37a01743-8a5c-42ea-9d9f-0d9e8b2009de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaac0ab583b344a189dfd1799386bd16', 'request_id': 'req-37a01743-8a5c-42ea-9d9f-0d9e8b2009de'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:58.247645 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-37a01743-8a5c-42ea-9d9f-0d9e8b2009de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaac0ab583b344a189dfd1799386bd16', 'request_id': 'req-37a01743-8a5c-42ea-9d9f-0d9e8b2009de'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:58.247821 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4856f8-4e6f-4704-a3ae-5527e793bdd3 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:58.309460 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee423c-38c8-4b8f-ac84-6c1c0a43feb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:58.310145 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee423c-38c8-4b8f-ac84-6c1c0a43feb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:58.310432 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee423c-38c8-4b8f-ac84-6c1c0a43feb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:58.310709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee423c-38c8-4b8f-ac84-6c1c0a43feb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:58.312004 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee423c-38c8-4b8f-ac84-6c1c0a43feb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:58.326793 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee423c-38c8-4b8f-ac84-6c1c0a43feb3 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:58.346385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-37a01743-8a5c-42ea-9d9f-0d9e8b2009de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaac0ab583b344a189dfd1799386bd16', 'request_id': 'req-37a01743-8a5c-42ea-9d9f-0d9e8b2009de'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:58.357243 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-37a01743-8a5c-42ea-9d9f-0d9e8b2009de admin admin] The token cache is being invalidate because project aaac0ab583b344a189dfd1799386bd16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:25:58.357533 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-37a01743-8a5c-42ea-9d9f-0d9e8b2009de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:58.358827 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-37a01743-8a5c-42ea-9d9f-0d9e8b2009de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:25:58.360479 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1059/2108] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:25:57 2022] DELETE /identity/v3/projects/aaac0ab583b344a189dfd1799386bd16 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:58.366526 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a90f42-3382-4927-826c-b8376a5459f5 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:58.366816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a90f42-3382-4927-826c-b8376a5459f5 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:58.367125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a90f42-3382-4927-826c-b8376a5459f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:58.384765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-00a90f42-3382-4927-826c-b8376a5459f5 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:58.385971 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-00a90f42-3382-4927-826c-b8376a5459f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:58.393947 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1060/2109] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:25:58.402685 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4572ac6-2d38-499c-9362-4a6be8868f51 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:58.403011 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4572ac6-2d38-499c-9362-4a6be8868f51 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:58.403293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4572ac6-2d38-499c-9362-4a6be8868f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:58.421887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-a4572ac6-2d38-499c-9362-4a6be8868f51 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:58.423112 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a4572ac6-2d38-499c-9362-4a6be8868f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:58.431346 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1061/2110] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:25:58.435001 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4572ac6-2d38-499c-9362-4a6be8868f51 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:58.439524 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1050/2111] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:25:58 2022] POST /identity/v3/users => generated 286 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:25:58.443525 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee423c-38c8-4b8f-ac84-6c1c0a43feb3 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:58.443934 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4572ac6-2d38-499c-9362-4a6be8868f51 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:58.445306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-f0ee423c-38c8-4b8f-ac84-6c1c0a43feb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:58.448664 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12903742-686f-431c-9553-e266a0011205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:58.449320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12903742-686f-431c-9553-e266a0011205 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:58.449608 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12903742-686f-431c-9553-e266a0011205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:58.449887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12903742-686f-431c-9553-e266a0011205 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:58.450621 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12903742-686f-431c-9553-e266a0011205 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:25:58.450963 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12903742-686f-431c-9553-e266a0011205 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:58.452605 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12903742-686f-431c-9553-e266a0011205 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:58.456834 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:58.459661 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee423c-38c8-4b8f-ac84-6c1c0a43feb3 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:58.464505 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c0ab5-a70d-4c51-9f14-e1a1afcba5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:58.465213 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c0ab5-a70d-4c51-9f14-e1a1afcba5d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:58.465502 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c0ab5-a70d-4c51-9f14-e1a1afcba5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:58.465784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c0ab5-a70d-4c51-9f14-e1a1afcba5d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:58.470274 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1062/2112] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:25:58.470435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c0ab5-a70d-4c51-9f14-e1a1afcba5d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:58.470435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c0ab5-a70d-4c51-9f14-e1a1afcba5d5 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:58.471086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12903742-686f-431c-9553-e266a0011205 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:58.479585 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1051/2113] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:25:58.482076 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12903742-686f-431c-9553-e266a0011205 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:58.484563 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c0ab5-a70d-4c51-9f14-e1a1afcba5d5 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:58.488465 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c54804-6dcd-402c-a5a1-455f56faca1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:58.489131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c54804-6dcd-402c-a5a1-455f56faca1c admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:58.489458 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c54804-6dcd-402c-a5a1-455f56faca1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:58.489745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c54804-6dcd-402c-a5a1-455f56faca1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:58.490450 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c54804-6dcd-402c-a5a1-455f56faca1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:58.492445 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c54804-6dcd-402c-a5a1-455f56faca1c admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:58.495887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c0ab5-a70d-4c51-9f14-e1a1afcba5d5 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:58.500747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482e327-d89f-4cb8-92ec-36cab6ab3205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:58.501542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482e327-d89f-4cb8-92ec-36cab6ab3205 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:58.501972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482e327-d89f-4cb8-92ec-36cab6ab3205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:58.502323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482e327-d89f-4cb8-92ec-36cab6ab3205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:58.503640 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0482e327-d89f-4cb8-92ec-36cab6ab3205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:58.505438 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0482e327-d89f-4cb8-92ec-36cab6ab3205 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:58.517323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1052/2114] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:25:58.521608 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482e327-d89f-4cb8-92ec-36cab6ab3205 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:58.533028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482e327-d89f-4cb8-92ec-36cab6ab3205 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:58.538096 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd18c7e-5663-413f-8b02-55a621e24de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:58.539028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd18c7e-5663-413f-8b02-55a621e24de8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:58.539408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd18c7e-5663-413f-8b02-55a621e24de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:58.539790 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd18c7e-5663-413f-8b02-55a621e24de8 admin admin] PATH_INFO: `/v3/projects/ad47e5b337424dfc97587aab8d898e29/users/aa94d47c041a4a6193dc2f9f78fb57c2/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:58.555162 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd18c7e-5663-413f-8b02-55a621e24de8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad47e5b337424dfc97587aab8d898e29, user_id=aa94d47c041a4a6193dc2f9f78fb57c2, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:58.557814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd18c7e-5663-413f-8b02-55a621e24de8 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:58.641915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1063/2115] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:25:58 2022] POST /identity/v3/projects => generated 381 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:25:58.645856 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c54804-6dcd-402c-a5a1-455f56faca1c admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:58.658051 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:58.664943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1053/2116] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:25:58 2022] PUT /identity/v3/projects/ad47e5b337424dfc97587aab8d898e29/users/aa94d47c041a4a6193dc2f9f78fb57c2/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:25:58.675036 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a23d4-8d17-426b-aaee-df90e8fb1709 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:58.675331 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a23d4-8d17-426b-aaee-df90e8fb1709 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:58.675611 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a23d4-8d17-426b-aaee-df90e8fb1709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:58.707228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-ed6a23d4-8d17-426b-aaee-df90e8fb1709 None None] MFA Rules not processed for user `aa94d47c041a4a6193dc2f9f78fb57c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:58.708435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-ed6a23d4-8d17-426b-aaee-df90e8fb1709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:58.714843 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c54804-6dcd-402c-a5a1-455f56faca1c admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:58.772881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df5b06-f7cd-4ab0-b748-0e0f2ac4a4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:58.773082 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1054/2117] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:25:58.773494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df5b06-f7cd-4ab0-b748-0e0f2ac4a4f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:58.773784 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df5b06-f7cd-4ab0-b748-0e0f2ac4a4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:58.774090 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df5b06-f7cd-4ab0-b748-0e0f2ac4a4f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:58.774796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df5b06-f7cd-4ab0-b748-0e0f2ac4a4f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:58.777064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df5b06-f7cd-4ab0-b748-0e0f2ac4a4f2 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:58.777611 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a23d4-8d17-426b-aaee-df90e8fb1709 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:58.788249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a23d4-8d17-426b-aaee-df90e8fb1709 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:58.792811 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c1b57-33f5-46c7-9eab-f373d199db39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:58.793447 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c1b57-33f5-46c7-9eab-f373d199db39 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:58.793735 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c1b57-33f5-46c7-9eab-f373d199db39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:58.794013 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c1b57-33f5-46c7-9eab-f373d199db39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:58.795029 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c1b57-33f5-46c7-9eab-f373d199db39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:58.797455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c1b57-33f5-46c7-9eab-f373d199db39 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:58.960279 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1064/2118] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:25:58 2022] POST /identity/v3/users => generated 291 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:25:58.966273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df5b06-f7cd-4ab0-b748-0e0f2ac4a4f2 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:58.976578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df5b06-f7cd-4ab0-b748-0e0f2ac4a4f2 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:58.981884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c4c55-d7e9-4a63-8619-48b92cfa6eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:58.982772 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c4c55-d7e9-4a63-8619-48b92cfa6eaa admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:58.983145 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c4c55-d7e9-4a63-8619-48b92cfa6eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:58.983700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c4c55-d7e9-4a63-8619-48b92cfa6eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:58.984968 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c4c55-d7e9-4a63-8619-48b92cfa6eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:58.986673 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c4c55-d7e9-4a63-8619-48b92cfa6eaa admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:58.991836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1055/2119] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:25:58 2022] POST /identity/v3/projects => generated 373 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:25:58.995632 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c1b57-33f5-46c7-9eab-f373d199db39 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:59.004604 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1065/2120] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:25:59.006224 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c4c55-d7e9-4a63-8619-48b92cfa6eaa admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:59.019908 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c4c55-d7e9-4a63-8619-48b92cfa6eaa admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:59.063614 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c1b57-33f5-46c7-9eab-f373d199db39 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:59.068367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a5b74-4cf9-49eb-a7d1-59eb08fcc986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:59.069047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a5b74-4cf9-49eb-a7d1-59eb08fcc986 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:59.069330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a5b74-4cf9-49eb-a7d1-59eb08fcc986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:59.069610 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a5b74-4cf9-49eb-a7d1-59eb08fcc986 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:59.070576 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a5b74-4cf9-49eb-a7d1-59eb08fcc986 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:59.072453 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a5b74-4cf9-49eb-a7d1-59eb08fcc986 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:59.082515 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a9d15c-eeb6-461c-91c6-3151e373d417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:59.084052 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9d15c-eeb6-461c-91c6-3151e373d417 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:59.084052 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9d15c-eeb6-461c-91c6-3151e373d417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:59.084052 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9d15c-eeb6-461c-91c6-3151e373d417 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:59.084684 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a9d15c-eeb6-461c-91c6-3151e373d417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:59.086332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a9d15c-eeb6-461c-91c6-3151e373d417 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:59.097147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1066/2121] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:25:59.101514 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a9d15c-eeb6-461c-91c6-3151e373d417 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:59.111768 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a9d15c-eeb6-461c-91c6-3151e373d417 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:59.116496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67f792-9c96-450b-adea-19c985074a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:59.117412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67f792-9c96-450b-adea-19c985074a6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:59.117706 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67f792-9c96-450b-adea-19c985074a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:59.118034 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67f792-9c96-450b-adea-19c985074a6f admin admin] PATH_INFO: `/v3/projects/3ae492d159ed4024bee8fabb97791f60/users/f91ecd0b270a46d9857df7b2729c93df/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:59.135344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa67f792-9c96-450b-adea-19c985074a6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ae492d159ed4024bee8fabb97791f60, user_id=f91ecd0b270a46d9857df7b2729c93df, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:59.138035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa67f792-9c96-450b-adea-19c985074a6f admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:59.185022 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1056/2122] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:25:58 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:25:59.188841 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a5b74-4cf9-49eb-a7d1-59eb08fcc986 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:59.199306 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a5b74-4cf9-49eb-a7d1-59eb08fcc986 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:59.205326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda2a8b-284e-4a54-97cf-47aa613d87c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:59.206063 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda2a8b-284e-4a54-97cf-47aa613d87c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:59.206362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda2a8b-284e-4a54-97cf-47aa613d87c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:59.206667 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda2a8b-284e-4a54-97cf-47aa613d87c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:59.207448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda2a8b-284e-4a54-97cf-47aa613d87c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:59.209094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda2a8b-284e-4a54-97cf-47aa613d87c6 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:59.222896 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1057/2123] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:25:59.227931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda2a8b-284e-4a54-97cf-47aa613d87c6 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:59.239329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda2a8b-284e-4a54-97cf-47aa613d87c6 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:59.246152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1067/2124] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:25:59 2022] PUT /identity/v3/projects/3ae492d159ed4024bee8fabb97791f60/users/f91ecd0b270a46d9857df7b2729c93df/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:25:59.263585 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6191659-c3d4-4dac-8e3a-2d721aa5e5f0 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:59.263967 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6191659-c3d4-4dac-8e3a-2d721aa5e5f0 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:59.264555 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6191659-c3d4-4dac-8e3a-2d721aa5e5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:59.274404 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:59.298619 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-e6191659-c3d4-4dac-8e3a-2d721aa5e5f0 None None] MFA Rules not processed for user `f91ecd0b270a46d9857df7b2729c93df`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:59.300778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-e6191659-c3d4-4dac-8e3a-2d721aa5e5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:59.335842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:25:59.346442 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5765a35-c290-40ab-b932-39333ba29d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:59.346442 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5765a35-c290-40ab-b932-39333ba29d36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:59.346442 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5765a35-c290-40ab-b932-39333ba29d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:59.346442 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5765a35-c290-40ab-b932-39333ba29d36 admin admin] PATH_INFO: `/v3/projects/192d44372cd84601aa7a58754aea617f/users/bb638a8144814b0a96e279ee70c35b00/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:59.371904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5765a35-c290-40ab-b932-39333ba29d36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=192d44372cd84601aa7a58754aea617f, user_id=bb638a8144814b0a96e279ee70c35b00, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:59.375739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5765a35-c290-40ab-b932-39333ba29d36 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:59.404173 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1068/2125] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:25:59.416218 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6191659-c3d4-4dac-8e3a-2d721aa5e5f0 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:59.420112 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-e6191659-c3d4-4dac-8e3a-2d721aa5e5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:59.462282 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1058/2126] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:25:59 2022] PUT /identity/v3/projects/192d44372cd84601aa7a58754aea617f/users/bb638a8144814b0a96e279ee70c35b00/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:25:59.466293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5765a35-c290-40ab-b932-39333ba29d36 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:59.490084 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6191659-c3d4-4dac-8e3a-2d721aa5e5f0 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:59.547565 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5765a35-c290-40ab-b932-39333ba29d36 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:59.553122 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6dc51-e18a-4ba4-8120-de73fab231ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:59.555567 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6dc51-e18a-4ba4-8120-de73fab231ce admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:59.555567 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6dc51-e18a-4ba4-8120-de73fab231ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:59.555567 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6dc51-e18a-4ba4-8120-de73fab231ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:59.555567 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a6dc51-e18a-4ba4-8120-de73fab231ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:59.556993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a6dc51-e18a-4ba4-8120-de73fab231ce admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:59.564476 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bef8e-7d45-4c62-9815-7c32c57c96d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:59.565142 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bef8e-7d45-4c62-9815-7c32c57c96d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:59.565434 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bef8e-7d45-4c62-9815-7c32c57c96d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:59.565719 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bef8e-7d45-4c62-9815-7c32c57c96d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:59.567262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1059/2127] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:25:59.571338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6dc51-e18a-4ba4-8120-de73fab231ce admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:59.581564 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6dc51-e18a-4ba4-8120-de73fab231ce admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:59.586440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a3db3-7802-4ea2-ab73-495d75121ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:59.587351 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a3db3-7802-4ea2-ab73-495d75121ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:59.587641 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a3db3-7802-4ea2-ab73-495d75121ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:59.587924 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a3db3-7802-4ea2-ab73-495d75121ed7 admin admin] PATH_INFO: `/v3/projects/192d44372cd84601aa7a58754aea617f/users/bb638a8144814b0a96e279ee70c35b00/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:59.589950 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a3db3-7802-4ea2-ab73-495d75121ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=192d44372cd84601aa7a58754aea617f, user_id=bb638a8144814b0a96e279ee70c35b00, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:59.593229 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a3db3-7802-4ea2-ab73-495d75121ed7 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:59.603431 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1060/2128] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:25:59 2022] PUT /identity/v3/projects/192d44372cd84601aa7a58754aea617f/users/bb638a8144814b0a96e279ee70c35b00/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:25:59.614099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c30c4-fd53-4a21-b951-3cb0b6b0a5d3 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:59.614629 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c30c4-fd53-4a21-b951-3cb0b6b0a5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:59.615108 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c30c4-fd53-4a21-b951-3cb0b6b0a5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:59.630164 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698bef8e-7d45-4c62-9815-7c32c57c96d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:59.630164 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698bef8e-7d45-4c62-9815-7c32c57c96d7 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:25:59.640269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1069/2129] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:25:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:25:59.645278 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-445c30c4-fd53-4a21-b951-3cb0b6b0a5d3 None None] MFA Rules not processed for user `bb638a8144814b0a96e279ee70c35b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:25:59.646507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-445c30c4-fd53-4a21-b951-3cb0b6b0a5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:25:59.661176 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bef8e-7d45-4c62-9815-7c32c57c96d7 service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:25:59.672354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bef8e-7d45-4c62-9815-7c32c57c96d7 service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:25:59.727976 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1061/2130] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:25:59 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:25:59.735821 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916ebfb-290f-457a-a9b3-21abb0ccba7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:25:59.736733 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916ebfb-290f-457a-a9b3-21abb0ccba7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:25:59.737024 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916ebfb-290f-457a-a9b3-21abb0ccba7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:25:59.739316 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916ebfb-290f-457a-a9b3-21abb0ccba7a admin admin] PATH_INFO: `/v3/users/f91ecd0b270a46d9857df7b2729c93df` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:25:59.739316 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2916ebfb-290f-457a-a9b3-21abb0ccba7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f91ecd0b270a46d9857df7b2729c93df)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:25:59.740839 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2916ebfb-290f-457a-a9b3-21abb0ccba7a admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:00.004987 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c30c4-fd53-4a21-b951-3cb0b6b0a5d3 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:00.070969 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c30c4-fd53-4a21-b951-3cb0b6b0a5d3 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:00.074586 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ebf613-64ce-49b2-b63c-233ba6609846 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:00.075280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ebf613-64ce-49b2-b63c-233ba6609846 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:00.075574 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ebf613-64ce-49b2-b63c-233ba6609846 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:00.075859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ebf613-64ce-49b2-b63c-233ba6609846 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:00.085636 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ebf613-64ce-49b2-b63c-233ba6609846 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:00.087195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ebf613-64ce-49b2-b63c-233ba6609846 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:00.097980 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1062/2131] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:00 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:26:00.174214 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2916ebfb-290f-457a-a9b3-21abb0ccba7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f91ecd0b270a46d9857df7b2729c93df'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:00.183016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2916ebfb-290f-457a-a9b3-21abb0ccba7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f91ecd0b270a46d9857df7b2729c93df'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:00.183862 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2916ebfb-290f-457a-a9b3-21abb0ccba7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f91ecd0b270a46d9857df7b2729c93df'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:00.184511 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2916ebfb-290f-457a-a9b3-21abb0ccba7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f91ecd0b270a46d9857df7b2729c93df'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:00.196536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2916ebfb-290f-457a-a9b3-21abb0ccba7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f91ecd0b270a46d9857df7b2729c93df'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:00.221585 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:00.290477 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1070/2132] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:25:59 2022] DELETE /identity/v3/users/f91ecd0b270a46d9857df7b2729c93df => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:26:00.295116 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec51cfc-ce80-4a0a-ac9d-61314b1a1cf5 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:00.295440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec51cfc-ce80-4a0a-ac9d-61314b1a1cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:00.295726 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec51cfc-ce80-4a0a-ac9d-61314b1a1cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:00.313126 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-7ec51cfc-ce80-4a0a-ac9d-61314b1a1cf5 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:00.314348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7ec51cfc-ce80-4a0a-ac9d-61314b1a1cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:00.377300 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1063/2133] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:26:00.388241 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916ebfb-290f-457a-a9b3-21abb0ccba7a admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:00.394037 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-2916ebfb-290f-457a-a9b3-21abb0ccba7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:00.418048 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:00.462885 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916ebfb-290f-457a-a9b3-21abb0ccba7a admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:00.466489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d14f07-3d6a-44c6-abf0-b12ca510c63b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:00.467178 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d14f07-3d6a-44c6-abf0-b12ca510c63b service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:00.467518 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d14f07-3d6a-44c6-abf0-b12ca510c63b service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:00.467958 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d14f07-3d6a-44c6-abf0-b12ca510c63b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:00.477328 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d14f07-3d6a-44c6-abf0-b12ca510c63b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:00.478817 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d14f07-3d6a-44c6-abf0-b12ca510c63b service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:00.489442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1071/2134] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:00.931843 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:00.978408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec51cfc-ce80-4a0a-ac9d-61314b1a1cf5 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:00.980972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7ec51cfc-ce80-4a0a-ac9d-61314b1a1cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:01.000669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec51cfc-ce80-4a0a-ac9d-61314b1a1cf5 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:01.007841 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1d31f-8e49-4e74-aa5a-05c9d29ae5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:01.008900 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1d31f-8e49-4e74-aa5a-05c9d29ae5b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:01.009325 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1d31f-8e49-4e74-aa5a-05c9d29ae5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:01.009740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1d31f-8e49-4e74-aa5a-05c9d29ae5b1 admin admin] PATH_INFO: `/v3/projects/3ae492d159ed4024bee8fabb97791f60` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:01.011109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1d31f-8e49-4e74-aa5a-05c9d29ae5b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ae492d159ed4024bee8fabb97791f60)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:01.014373 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1d31f-8e49-4e74-aa5a-05c9d29ae5b1 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:01.164023 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:01.164023 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-54a1d31f-8e49-4e74-aa5a-05c9d29ae5b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ae492d159ed4024bee8fabb97791f60', 'request_id': 'req-54a1d31f-8e49-4e74-aa5a-05c9d29ae5b1'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:01.169398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-54a1d31f-8e49-4e74-aa5a-05c9d29ae5b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ae492d159ed4024bee8fabb97791f60', 'request_id': 'req-54a1d31f-8e49-4e74-aa5a-05c9d29ae5b1'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:01.175878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-54a1d31f-8e49-4e74-aa5a-05c9d29ae5b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ae492d159ed4024bee8fabb97791f60', 'request_id': 'req-54a1d31f-8e49-4e74-aa5a-05c9d29ae5b1'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:01.320607 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-54a1d31f-8e49-4e74-aa5a-05c9d29ae5b1 admin admin] The token cache is being invalidate because project 3ae492d159ed4024bee8fabb97791f60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:26:01.320808 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d14f07-3d6a-44c6-abf0-b12ca510c63b service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:01.321797 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-54a1d31f-8e49-4e74-aa5a-05c9d29ae5b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:01.322667 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-54a1d31f-8e49-4e74-aa5a-05c9d29ae5b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:01.323428 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-c1d14f07-3d6a-44c6-abf0-b12ca510c63b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:01.333367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1064/2135] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:26:00 2022] DELETE /identity/v3/projects/3ae492d159ed4024bee8fabb97791f60 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:01.336966 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a203e27-3886-46c6-a345-82a188923f06 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:01.337266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a203e27-3886-46c6-a345-82a188923f06 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:01.337548 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a203e27-3886-46c6-a345-82a188923f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:01.354077 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d14f07-3d6a-44c6-abf0-b12ca510c63b service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:01.366676 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-2a203e27-3886-46c6-a345-82a188923f06 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:01.367949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-2a203e27-3886-46c6-a345-82a188923f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:01.429364 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8af08-a408-430e-86ef-5bc777a00015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:01.430324 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8af08-a408-430e-86ef-5bc777a00015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:01.430900 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8af08-a408-430e-86ef-5bc777a00015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:01.431366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8af08-a408-430e-86ef-5bc777a00015 admin admin] PATH_INFO: `/v3/users/aa94d47c041a4a6193dc2f9f78fb57c2` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:01.432854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8af08-a408-430e-86ef-5bc777a00015 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa94d47c041a4a6193dc2f9f78fb57c2)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:01.435662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8af08-a408-430e-86ef-5bc777a00015 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:01.503455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1065/2136] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:26:01.511457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a8fcb-d00b-4f5f-824d-5e1b0b342826 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:01.511757 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a8fcb-d00b-4f5f-824d-5e1b0b342826 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:01.512045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a8fcb-d00b-4f5f-824d-5e1b0b342826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:01.541506 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-9d3a8fcb-d00b-4f5f-824d-5e1b0b342826 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:01.542786 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-9d3a8fcb-d00b-4f5f-824d-5e1b0b342826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:01.560501 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1066/2137] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:01.565103 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a8fcb-d00b-4f5f-824d-5e1b0b342826 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:01.574605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a8fcb-d00b-4f5f-824d-5e1b0b342826 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:01.581154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ddc0a-91b1-4a1f-943a-e855885a5198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:01.581931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ddc0a-91b1-4a1f-943a-e855885a5198 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:01.589094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ddc0a-91b1-4a1f-943a-e855885a5198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:01.589094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ddc0a-91b1-4a1f-943a-e855885a5198 admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:01.589094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ddc0a-91b1-4a1f-943a-e855885a5198 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:26:01.589094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ddc0a-91b1-4a1f-943a-e855885a5198 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:01.589094 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ddc0a-91b1-4a1f-943a-e855885a5198 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:01.600481 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1067/2138] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:26:01.604685 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ddc0a-91b1-4a1f-943a-e855885a5198 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:01.615734 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ddc0a-91b1-4a1f-943a-e855885a5198 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:01.621499 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443aab3-66c3-4305-9925-eafeb0445b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:01.621678 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443aab3-66c3-4305-9925-eafeb0445b5c admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:01.622005 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443aab3-66c3-4305-9925-eafeb0445b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:01.622390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443aab3-66c3-4305-9925-eafeb0445b5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:01.623174 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443aab3-66c3-4305-9925-eafeb0445b5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:01.626406 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443aab3-66c3-4305-9925-eafeb0445b5c admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:01.751650 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7ff8af08-a408-430e-86ef-5bc777a00015 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa94d47c041a4a6193dc2f9f78fb57c2'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:01.761530 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7ff8af08-a408-430e-86ef-5bc777a00015 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa94d47c041a4a6193dc2f9f78fb57c2'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:01.762866 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7ff8af08-a408-430e-86ef-5bc777a00015 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa94d47c041a4a6193dc2f9f78fb57c2'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:01.764297 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7ff8af08-a408-430e-86ef-5bc777a00015 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa94d47c041a4a6193dc2f9f78fb57c2'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:01.778347 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7ff8af08-a408-430e-86ef-5bc777a00015 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa94d47c041a4a6193dc2f9f78fb57c2'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:01.823998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1068/2139] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:26:01 2022] POST /identity/v3/projects => generated 373 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:26:01.828264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443aab3-66c3-4305-9925-eafeb0445b5c admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:01.830089 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7443aab3-66c3-4305-9925-eafeb0445b5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:01.916609 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1072/2140] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:26:01 2022] DELETE /identity/v3/users/aa94d47c041a4a6193dc2f9f78fb57c2 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:01.921757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be146b9-8f02-441d-ab2e-7c3ccd18b2a1 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:01.922590 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be146b9-8f02-441d-ab2e-7c3ccd18b2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:01.922996 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be146b9-8f02-441d-ab2e-7c3ccd18b2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:01.930525 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443aab3-66c3-4305-9925-eafeb0445b5c admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:01.943048 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-9be146b9-8f02-441d-ab2e-7c3ccd18b2a1 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:01.944399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-9be146b9-8f02-441d-ab2e-7c3ccd18b2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:01.995925 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed719d3-b039-43a7-ad75-621e4faec1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:01.996907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed719d3-b039-43a7-ad75-621e4faec1f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:01.997274 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed719d3-b039-43a7-ad75-621e4faec1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:01.997632 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed719d3-b039-43a7-ad75-621e4faec1f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:01.998503 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed719d3-b039-43a7-ad75-621e4faec1f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:02.001400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed719d3-b039-43a7-ad75-621e4faec1f0 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:02.048833 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1073/2141] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:26:02.075110 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be146b9-8f02-441d-ab2e-7c3ccd18b2a1 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:02.075110 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-9be146b9-8f02-441d-ab2e-7c3ccd18b2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:02.132434 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1069/2142] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:26:01 2022] POST /identity/v3/users => generated 287 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:26:02.136665 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed719d3-b039-43a7-ad75-621e4faec1f0 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:02.146300 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be146b9-8f02-441d-ab2e-7c3ccd18b2a1 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:02.146761 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed719d3-b039-43a7-ad75-621e4faec1f0 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:02.155309 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55db1b-dd6c-42df-a3e9-0be9fbe9af89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:02.158305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55db1b-dd6c-42df-a3e9-0be9fbe9af89 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:02.158612 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55db1b-dd6c-42df-a3e9-0be9fbe9af89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:02.159057 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55db1b-dd6c-42df-a3e9-0be9fbe9af89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:02.159176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98d30d-2552-43fb-bc97-221c320098fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:02.165043 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98d30d-2552-43fb-bc97-221c320098fb admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:02.165043 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98d30d-2552-43fb-bc97-221c320098fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:02.165043 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98d30d-2552-43fb-bc97-221c320098fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:02.165043 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c98d30d-2552-43fb-bc97-221c320098fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:02.165043 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c98d30d-2552-43fb-bc97-221c320098fb admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:02.170074 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55db1b-dd6c-42df-a3e9-0be9fbe9af89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:02.172253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55db1b-dd6c-42df-a3e9-0be9fbe9af89 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:02.176850 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1070/2143] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:26:02.180683 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98d30d-2552-43fb-bc97-221c320098fb admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:02.182769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1074/2144] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:02.190603 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98d30d-2552-43fb-bc97-221c320098fb admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:02.195371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2209a9-2558-478a-96bc-c5bedd2d1e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:02.196040 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2209a9-2558-478a-96bc-c5bedd2d1e11 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:02.196333 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2209a9-2558-478a-96bc-c5bedd2d1e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:02.196619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2209a9-2558-478a-96bc-c5bedd2d1e11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:02.197352 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2209a9-2558-478a-96bc-c5bedd2d1e11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:02.199179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2209a9-2558-478a-96bc-c5bedd2d1e11 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:02.212832 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1071/2145] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:26:02.214013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55db1b-dd6c-42df-a3e9-0be9fbe9af89 service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:02.224219 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55db1b-dd6c-42df-a3e9-0be9fbe9af89 service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:02.229234 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbea1f-553e-489e-8697-a46e292b43da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:02.230116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbea1f-553e-489e-8697-a46e292b43da admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:02.230411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbea1f-553e-489e-8697-a46e292b43da admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:02.230703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbea1f-553e-489e-8697-a46e292b43da admin admin] PATH_INFO: `/v3/projects/11d4f13492884e99a4ad6014bb470acc/users/3dfe3bb587ec4adfab8245da46f9b24b/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:02.254356 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbea1f-553e-489e-8697-a46e292b43da admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d4f13492884e99a4ad6014bb470acc, user_id=3dfe3bb587ec4adfab8245da46f9b24b, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:02.257122 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbea1f-553e-489e-8697-a46e292b43da admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:02.352316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:02.398599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1075/2146] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:26:02 2022] PUT /identity/v3/projects/11d4f13492884e99a4ad6014bb470acc/users/3dfe3bb587ec4adfab8245da46f9b24b/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:02.410251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c42e2-af31-4bd4-97b8-b726043b8aaa None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:02.411087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c42e2-af31-4bd4-97b8-b726043b8aaa None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:02.411550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c42e2-af31-4bd4-97b8-b726043b8aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:02.417695 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:02.450120 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-5a7c42e2-af31-4bd4-97b8-b726043b8aaa None None] MFA Rules not processed for user `3dfe3bb587ec4adfab8245da46f9b24b`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:02.452321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-5a7c42e2-af31-4bd4-97b8-b726043b8aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:02.578950 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1072/2147] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:02.588907 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbea1f-553e-489e-8697-a46e292b43da admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:02.591809 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d1dbea1f-553e-489e-8697-a46e292b43da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:02.663421 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c42e2-af31-4bd4-97b8-b726043b8aaa None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:02.665571 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-5a7c42e2-af31-4bd4-97b8-b726043b8aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:02.677978 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbea1f-553e-489e-8697-a46e292b43da admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:02.682109 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b2aff-772e-4f50-b4f9-684048c2af67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:02.683093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b2aff-772e-4f50-b4f9-684048c2af67 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:02.683334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b2aff-772e-4f50-b4f9-684048c2af67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:02.684214 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b2aff-772e-4f50-b4f9-684048c2af67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:02.696243 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b2aff-772e-4f50-b4f9-684048c2af67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:02.697908 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b2aff-772e-4f50-b4f9-684048c2af67 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:02.709269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1076/2148] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:02.755044 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c42e2-af31-4bd4-97b8-b726043b8aaa None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:02.761895 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6a3b2-82df-45cd-be4d-23d816b5cfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:02.763040 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6a3b2-82df-45cd-be4d-23d816b5cfb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:02.763403 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6a3b2-82df-45cd-be4d-23d816b5cfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:02.763755 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6a3b2-82df-45cd-be4d-23d816b5cfb4 admin admin] PATH_INFO: `/v3/projects/ad47e5b337424dfc97587aab8d898e29` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:02.766517 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6a3b2-82df-45cd-be4d-23d816b5cfb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad47e5b337424dfc97587aab8d898e29)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:02.769265 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6a3b2-82df-45cd-be4d-23d816b5cfb4 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:02.933291 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e4f6a3b2-82df-45cd-be4d-23d816b5cfb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad47e5b337424dfc97587aab8d898e29', 'request_id': 'req-e4f6a3b2-82df-45cd-be4d-23d816b5cfb4'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:02.944946 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e4f6a3b2-82df-45cd-be4d-23d816b5cfb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad47e5b337424dfc97587aab8d898e29', 'request_id': 'req-e4f6a3b2-82df-45cd-be4d-23d816b5cfb4'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:02.950592 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e4f6a3b2-82df-45cd-be4d-23d816b5cfb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad47e5b337424dfc97587aab8d898e29', 'request_id': 'req-e4f6a3b2-82df-45cd-be4d-23d816b5cfb4'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:03.062735 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e4f6a3b2-82df-45cd-be4d-23d816b5cfb4 admin admin] The token cache is being invalidate because project ad47e5b337424dfc97587aab8d898e29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:26:03.063041 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e4f6a3b2-82df-45cd-be4d-23d816b5cfb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:03.063678 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e4f6a3b2-82df-45cd-be4d-23d816b5cfb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:03.065277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1073/2149] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:26:02 2022] DELETE /identity/v3/projects/ad47e5b337424dfc97587aab8d898e29 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:03.069184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b2aff-772e-4f50-b4f9-684048c2af67 service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:03.080562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b2aff-772e-4f50-b4f9-684048c2af67 service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:03.142301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c04ce-e8a4-48af-b40a-e41c32436eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:03.143520 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c04ce-e8a4-48af-b40a-e41c32436eeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:03.143870 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c04ce-e8a4-48af-b40a-e41c32436eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:03.144278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c04ce-e8a4-48af-b40a-e41c32436eeb admin admin] PATH_INFO: `/v3/users/bb638a8144814b0a96e279ee70c35b00` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:03.145625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c04ce-e8a4-48af-b40a-e41c32436eeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb638a8144814b0a96e279ee70c35b00)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:03.147808 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c04ce-e8a4-48af-b40a-e41c32436eeb admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:03.521489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-210c04ce-e8a4-48af-b40a-e41c32436eeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb638a8144814b0a96e279ee70c35b00'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:03.528602 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-210c04ce-e8a4-48af-b40a-e41c32436eeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb638a8144814b0a96e279ee70c35b00'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:03.529266 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-210c04ce-e8a4-48af-b40a-e41c32436eeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb638a8144814b0a96e279ee70c35b00'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:03.529878 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-210c04ce-e8a4-48af-b40a-e41c32436eeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb638a8144814b0a96e279ee70c35b00'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:03.541355 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-210c04ce-e8a4-48af-b40a-e41c32436eeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb638a8144814b0a96e279ee70c35b00'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:03.590067 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae4179-897a-4def-b916-77396ef8b52b None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:03.590067 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae4179-897a-4def-b916-77396ef8b52b None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:03.590067 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae4179-897a-4def-b916-77396ef8b52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:03.612950 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-8fae4179-897a-4def-b916-77396ef8b52b None None] MFA Rules not processed for user `cfbaba84402d4c31bafcd12133d74c03`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:03.614894 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-8fae4179-897a-4def-b916-77396ef8b52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:03.616518 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1077/2150] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:26:03 2022] DELETE /identity/v3/users/bb638a8144814b0a96e279ee70c35b00 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:03.704241 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1074/2151] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jun 23 09:26:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:03.766510 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c04ce-e8a4-48af-b40a-e41c32436eeb admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:03.770452 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-210c04ce-e8a4-48af-b40a-e41c32436eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:03.850068 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c04ce-e8a4-48af-b40a-e41c32436eeb admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:03.855009 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42796b-5e6a-4727-8ab3-d052dc5e3a6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b1331b4f0924bb8b137a22a33c9d130', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:03.855667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42796b-5e6a-4727-8ab3-d052dc5e3a6d service swift] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:03.855969 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42796b-5e6a-4727-8ab3-d052dc5e3a6d service swift] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:03.856249 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42796b-5e6a-4727-8ab3-d052dc5e3a6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:03.865492 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42796b-5e6a-4727-8ab3-d052dc5e3a6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:03.867360 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42796b-5e6a-4727-8ab3-d052dc5e3a6d service swift] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:03.876606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1078/2152] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jun 23 09:26:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 09:26:04.355379 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae4179-897a-4def-b916-77396ef8b52b None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:04.357114 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-8fae4179-897a-4def-b916-77396ef8b52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:04.459700 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae4179-897a-4def-b916-77396ef8b52b None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:04.490633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77428fad-5907-43d6-a452-ed39c4a7fc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:04.493646 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77428fad-5907-43d6-a452-ed39c4a7fc9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:04.493948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77428fad-5907-43d6-a452-ed39c4a7fc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:04.494231 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77428fad-5907-43d6-a452-ed39c4a7fc9f admin admin] PATH_INFO: `/v3/projects/192d44372cd84601aa7a58754aea617f` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:04.506394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77428fad-5907-43d6-a452-ed39c4a7fc9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=192d44372cd84601aa7a58754aea617f)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:04.515157 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77428fad-5907-43d6-a452-ed39c4a7fc9f admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:04.750866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-77428fad-5907-43d6-a452-ed39c4a7fc9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '192d44372cd84601aa7a58754aea617f', 'request_id': 'req-77428fad-5907-43d6-a452-ed39c4a7fc9f'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:04.758754 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-77428fad-5907-43d6-a452-ed39c4a7fc9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '192d44372cd84601aa7a58754aea617f', 'request_id': 'req-77428fad-5907-43d6-a452-ed39c4a7fc9f'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:04.777402 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-77428fad-5907-43d6-a452-ed39c4a7fc9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '192d44372cd84601aa7a58754aea617f', 'request_id': 'req-77428fad-5907-43d6-a452-ed39c4a7fc9f'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:04.930103 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-77428fad-5907-43d6-a452-ed39c4a7fc9f admin admin] The token cache is being invalidate because project 192d44372cd84601aa7a58754aea617f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:26:04.930642 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-77428fad-5907-43d6-a452-ed39c4a7fc9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:04.935095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-77428fad-5907-43d6-a452-ed39c4a7fc9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:04.940529 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1075/2153] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:26:04 2022] DELETE /identity/v3/projects/192d44372cd84601aa7a58754aea617f => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:04.942030 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf3450-9d0c-411e-ab3a-4318df96928a None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:04.942855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf3450-9d0c-411e-ab3a-4318df96928a None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:04.943337 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf3450-9d0c-411e-ab3a-4318df96928a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:04.982612 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-9abf3450-9d0c-411e-ab3a-4318df96928a None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:04.984961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-9abf3450-9d0c-411e-ab3a-4318df96928a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:05.066683 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1079/2154] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:26:05.080022 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce424612-e908-47fd-ab0f-d3d54868f318 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:05.080366 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce424612-e908-47fd-ab0f-d3d54868f318 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:05.080600 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce424612-e908-47fd-ab0f-d3d54868f318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:05.138242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-ce424612-e908-47fd-ab0f-d3d54868f318 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:05.140319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-ce424612-e908-47fd-ab0f-d3d54868f318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:05.154973 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1076/2155] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:05.190761 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf3450-9d0c-411e-ab3a-4318df96928a None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:05.190761 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf3450-9d0c-411e-ab3a-4318df96928a None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:05.206432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce3be4-f90d-4b4a-b4d7-11c7c6913e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:05.207443 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce3be4-f90d-4b4a-b4d7-11c7c6913e61 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:05.207903 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce3be4-f90d-4b4a-b4d7-11c7c6913e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:05.214531 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce3be4-f90d-4b4a-b4d7-11c7c6913e61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:05.216347 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce3be4-f90d-4b4a-b4d7-11c7c6913e61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:26:05.216856 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce3be4-f90d-4b4a-b4d7-11c7c6913e61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:05.218995 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce3be4-f90d-4b4a-b4d7-11c7c6913e61 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:05.236816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1080/2156] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:26:05.242518 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce424612-e908-47fd-ab0f-d3d54868f318 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:05.262557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce424612-e908-47fd-ab0f-d3d54868f318 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:05.270016 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3a8d0-7a5e-44a9-a79a-b230556226f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:05.270814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3a8d0-7a5e-44a9-a79a-b230556226f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:05.271201 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3a8d0-7a5e-44a9-a79a-b230556226f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:05.271545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3a8d0-7a5e-44a9-a79a-b230556226f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:05.272435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3a8d0-7a5e-44a9-a79a-b230556226f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:05.274685 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3a8d0-7a5e-44a9-a79a-b230556226f8 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:05.687083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:05.694120 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1077/2157] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:26:05 2022] POST /identity/v3/projects => generated 399 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:26:05.698305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce3be4-f90d-4b4a-b4d7-11c7c6913e61 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:05.727566 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce3be4-f90d-4b4a-b4d7-11c7c6913e61 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:05.777770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:05.944408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92527d2d-e80b-4c6a-8b34-b1e6f5625cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:05.946186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92527d2d-e80b-4c6a-8b34-b1e6f5625cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:05.946482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92527d2d-e80b-4c6a-8b34-b1e6f5625cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:05.947522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92527d2d-e80b-4c6a-8b34-b1e6f5625cc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:05.948348 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92527d2d-e80b-4c6a-8b34-b1e6f5625cc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:05.950793 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92527d2d-e80b-4c6a-8b34-b1e6f5625cc2 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:06.088116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1081/2158] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:26:05 2022] POST /identity/v3/users => generated 300 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:26:06.092262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3a8d0-7a5e-44a9-a79a-b230556226f8 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:06.127141 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3a8d0-7a5e-44a9-a79a-b230556226f8 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:06.146707 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4393c3-3467-44d1-8a09-81d6c82b0cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:06.147531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4393c3-3467-44d1-8a09-81d6c82b0cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:06.147934 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4393c3-3467-44d1-8a09-81d6c82b0cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:06.148395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4393c3-3467-44d1-8a09-81d6c82b0cf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:06.149737 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4393c3-3467-44d1-8a09-81d6c82b0cf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:06.152143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4393c3-3467-44d1-8a09-81d6c82b0cf7 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:06.185220 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1078/2159] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:26:06.188238 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92527d2d-e80b-4c6a-8b34-b1e6f5625cc2 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:06.203806 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92527d2d-e80b-4c6a-8b34-b1e6f5625cc2 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:06.213311 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d1fa9-dbcb-4f25-8e3b-9865c051b4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:06.214160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d1fa9-dbcb-4f25-8e3b-9865c051b4ff admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:06.214524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d1fa9-dbcb-4f25-8e3b-9865c051b4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:06.216165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d1fa9-dbcb-4f25-8e3b-9865c051b4ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:06.217281 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150d1fa9-dbcb-4f25-8e3b-9865c051b4ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:06.219435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150d1fa9-dbcb-4f25-8e3b-9865c051b4ff admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:06.235628 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1082/2160] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:26:06.240010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4393c3-3467-44d1-8a09-81d6c82b0cf7 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:06.265518 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4393c3-3467-44d1-8a09-81d6c82b0cf7 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:06.275294 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce5f75-1643-4850-986b-b1f658ddfa80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:06.276649 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce5f75-1643-4850-986b-b1f658ddfa80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:06.277075 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce5f75-1643-4850-986b-b1f658ddfa80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:06.277658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce5f75-1643-4850-986b-b1f658ddfa80 admin admin] PATH_INFO: `/v3/projects/a3469e42573a465d9520c90c06eb78d0/users/69db77e4b32847249a6289fc48e75bef/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:06.303297 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce5f75-1643-4850-986b-b1f658ddfa80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3469e42573a465d9520c90c06eb78d0, user_id=69db77e4b32847249a6289fc48e75bef, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:06.306527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce5f75-1643-4850-986b-b1f658ddfa80 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:06.354837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d1fa9-dbcb-4f25-8e3b-9865c051b4ff admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:06.357651 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-150d1fa9-dbcb-4f25-8e3b-9865c051b4ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:06.375007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d1fa9-dbcb-4f25-8e3b-9865c051b4ff admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:06.380254 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e5d20-3bb5-4fa7-ba94-666fd550494a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:06.381543 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e5d20-3bb5-4fa7-ba94-666fd550494a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:06.381811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e5d20-3bb5-4fa7-ba94-666fd550494a admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:06.382123 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e5d20-3bb5-4fa7-ba94-666fd550494a admin admin] PATH_INFO: `/v3/users/363d9d9e5ce145dc996309af1a60bb2a` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:06.383422 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12e5d20-3bb5-4fa7-ba94-666fd550494a admin admin] RBAC: Authorizing `identity:delete_user(user_id=363d9d9e5ce145dc996309af1a60bb2a)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:06.385627 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12e5d20-3bb5-4fa7-ba94-666fd550494a admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:06.414871 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1079/2161] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:26:06 2022] PUT /identity/v3/projects/a3469e42573a465d9520c90c06eb78d0/users/69db77e4b32847249a6289fc48e75bef/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:06.426747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0a051-79c7-472d-befe-32a2efba5f99 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:06.427408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0a051-79c7-472d-befe-32a2efba5f99 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:06.428377 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0a051-79c7-472d-befe-32a2efba5f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:06.478810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-7ef0a051-79c7-472d-befe-32a2efba5f99 None None] MFA Rules not processed for user `69db77e4b32847249a6289fc48e75bef`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:06.480026 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7ef0a051-79c7-472d-befe-32a2efba5f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:06.618044 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1080/2162] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:06 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:06.622141 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0a051-79c7-472d-befe-32a2efba5f99 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:06.754878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0a051-79c7-472d-befe-32a2efba5f99 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:06.760835 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab3546-c49c-4ece-8efe-c05246a0eee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:06.761541 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab3546-c49c-4ece-8efe-c05246a0eee1 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:06.761827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab3546-c49c-4ece-8efe-c05246a0eee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:06.762157 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab3546-c49c-4ece-8efe-c05246a0eee1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:06.762874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab3546-c49c-4ece-8efe-c05246a0eee1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:06.765930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab3546-c49c-4ece-8efe-c05246a0eee1 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:06.769991 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-c12e5d20-3bb5-4fa7-ba94-666fd550494a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '363d9d9e5ce145dc996309af1a60bb2a'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:06.781791 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-c12e5d20-3bb5-4fa7-ba94-666fd550494a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '363d9d9e5ce145dc996309af1a60bb2a'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:06.786658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-c12e5d20-3bb5-4fa7-ba94-666fd550494a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '363d9d9e5ce145dc996309af1a60bb2a'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:06.786658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-c12e5d20-3bb5-4fa7-ba94-666fd550494a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '363d9d9e5ce145dc996309af1a60bb2a'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:06.801733 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-c12e5d20-3bb5-4fa7-ba94-666fd550494a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '363d9d9e5ce145dc996309af1a60bb2a'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:06.881842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1081/2163] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:26:06 2022] POST /identity/v3/projects => generated 399 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:26:06.886087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab3546-c49c-4ece-8efe-c05246a0eee1 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:06.888671 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-3aab3546-c49c-4ece-8efe-c05246a0eee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:06.940519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1083/2164] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:26:06 2022] DELETE /identity/v3/users/363d9d9e5ce145dc996309af1a60bb2a => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:06.944813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a9567-ee5b-4bb0-8720-c83f4e924b86 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:06.945792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a9567-ee5b-4bb0-8720-c83f4e924b86 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:06.945792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a9567-ee5b-4bb0-8720-c83f4e924b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:06.979081 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-ba2a9567-ee5b-4bb0-8720-c83f4e924b86 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:06.981570 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ba2a9567-ee5b-4bb0-8720-c83f4e924b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:07.042102 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab3546-c49c-4ece-8efe-c05246a0eee1 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:07.102683 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1084/2165] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:07.119784 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a9567-ee5b-4bb0-8720-c83f4e924b86 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:07.124976 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ba2a9567-ee5b-4bb0-8720-c83f4e924b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:07.168219 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf0439-77c1-4c06-856b-991c0dac5c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:07.168971 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf0439-77c1-4c06-856b-991c0dac5c75 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:07.169570 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf0439-77c1-4c06-856b-991c0dac5c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:07.169940 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf0439-77c1-4c06-856b-991c0dac5c75 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:07.172317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf0439-77c1-4c06-856b-991c0dac5c75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:07.174400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf0439-77c1-4c06-856b-991c0dac5c75 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:07.266871 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a9567-ee5b-4bb0-8720-c83f4e924b86 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:07.271216 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080d729-a262-4261-a289-636177150cd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:07.272663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080d729-a262-4261-a289-636177150cd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:07.272663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080d729-a262-4261-a289-636177150cd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:07.272663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080d729-a262-4261-a289-636177150cd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:07.292775 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080d729-a262-4261-a289-636177150cd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:07.294750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080d729-a262-4261-a289-636177150cd6 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:07.304759 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1082/2166] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:26:06 2022] POST /identity/v3/users => generated 306 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:26:07.310196 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf0439-77c1-4c06-856b-991c0dac5c75 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:07.311764 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1085/2167] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:07.336772 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf0439-77c1-4c06-856b-991c0dac5c75 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:07.349068 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a608f-1f0c-42d3-9ed4-a8cae8508296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:07.349979 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a608f-1f0c-42d3-9ed4-a8cae8508296 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:07.353822 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a608f-1f0c-42d3-9ed4-a8cae8508296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:07.353966 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a608f-1f0c-42d3-9ed4-a8cae8508296 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:07.354650 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a608f-1f0c-42d3-9ed4-a8cae8508296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:07.356873 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a608f-1f0c-42d3-9ed4-a8cae8508296 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:07.400324 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080d729-a262-4261-a289-636177150cd6 service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:07.401339 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1083/2168] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:26:07.458697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080d729-a262-4261-a289-636177150cd6 service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:07.488382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbfb66-b72c-42ed-a174-c9528ee0eb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:07.488382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbfb66-b72c-42ed-a174-c9528ee0eb32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:07.488382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbfb66-b72c-42ed-a174-c9528ee0eb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:07.488382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbfb66-b72c-42ed-a174-c9528ee0eb32 admin admin] PATH_INFO: `/v3/projects/f38a92d89edd40a9bf2c9f7a3101f618/users/a6d9aadce6e1409aa7b9e6cbcf5444ef/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:07.511986 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:07.535461 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbfb66-b72c-42ed-a174-c9528ee0eb32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f38a92d89edd40a9bf2c9f7a3101f618, user_id=a6d9aadce6e1409aa7b9e6cbcf5444ef, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:07.538212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbfb66-b72c-42ed-a174-c9528ee0eb32 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:07.723041 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1086/2169] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:26:07 2022] PUT /identity/v3/projects/f38a92d89edd40a9bf2c9f7a3101f618/users/a6d9aadce6e1409aa7b9e6cbcf5444ef/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:26:07.724385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a608f-1f0c-42d3-9ed4-a8cae8508296 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:07.753925 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a608f-1f0c-42d3-9ed4-a8cae8508296 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:07.868324 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32764125-9842-4b7c-a5c8-e68bc1b1c0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:07.868324 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32764125-9842-4b7c-a5c8-e68bc1b1c0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:07.868324 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32764125-9842-4b7c-a5c8-e68bc1b1c0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:07.868324 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32764125-9842-4b7c-a5c8-e68bc1b1c0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:07.868714 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32764125-9842-4b7c-a5c8-e68bc1b1c0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:07.868714 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32764125-9842-4b7c-a5c8-e68bc1b1c0e6 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:07.900967 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1084/2170] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:26:07.909049 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbfb66-b72c-42ed-a174-c9528ee0eb32 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:07.931841 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbfb66-b72c-42ed-a174-c9528ee0eb32 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:07.937634 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf3a78-2a9a-44b8-9c70-8a1488988f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:07.939032 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf3a78-2a9a-44b8-9c70-8a1488988f5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:07.939541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf3a78-2a9a-44b8-9c70-8a1488988f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:07.940029 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf3a78-2a9a-44b8-9c70-8a1488988f5a admin admin] PATH_INFO: `/v3/projects/f38a92d89edd40a9bf2c9f7a3101f618/users/a6d9aadce6e1409aa7b9e6cbcf5444ef/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:07.942536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf3a78-2a9a-44b8-9c70-8a1488988f5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f38a92d89edd40a9bf2c9f7a3101f618, user_id=a6d9aadce6e1409aa7b9e6cbcf5444ef, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:07.946376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf3a78-2a9a-44b8-9c70-8a1488988f5a admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:07.974102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1087/2171] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:26:07 2022] PUT /identity/v3/projects/f38a92d89edd40a9bf2c9f7a3101f618/users/a6d9aadce6e1409aa7b9e6cbcf5444ef/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:07.975618 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642242e0-966d-4bae-9610-ce3b892f768e None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:07.975924 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642242e0-966d-4bae-9610-ce3b892f768e None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:07.976217 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642242e0-966d-4bae-9610-ce3b892f768e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:08.016725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-642242e0-966d-4bae-9610-ce3b892f768e None None] MFA Rules not processed for user `a6d9aadce6e1409aa7b9e6cbcf5444ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:08.017981 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-642242e0-966d-4bae-9610-ce3b892f768e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:08.157258 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1085/2172] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:07 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:08.169770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf3a78-2a9a-44b8-9c70-8a1488988f5a admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:08.185495 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5ebf3a78-2a9a-44b8-9c70-8a1488988f5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:08.231576 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642242e0-966d-4bae-9610-ce3b892f768e None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:08.248955 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:08.250940 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-642242e0-966d-4bae-9610-ce3b892f768e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:08.321706 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf3a78-2a9a-44b8-9c70-8a1488988f5a admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:08.369240 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290d2bc-ac64-4951-b7a9-544fbeefdd18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:08.369240 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290d2bc-ac64-4951-b7a9-544fbeefdd18 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:08.369240 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290d2bc-ac64-4951-b7a9-544fbeefdd18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:08.369240 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290d2bc-ac64-4951-b7a9-544fbeefdd18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:08.369693 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290d2bc-ac64-4951-b7a9-544fbeefdd18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:08.369693 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290d2bc-ac64-4951-b7a9-544fbeefdd18 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:08.369693 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1088/2173] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:08 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:08.391902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642242e0-966d-4bae-9610-ce3b892f768e None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:08.396878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1cf6f-894b-43f6-b742-2c763375987c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:08.397760 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1cf6f-894b-43f6-b742-2c763375987c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:08.398337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1cf6f-894b-43f6-b742-2c763375987c admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:08.398756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1cf6f-894b-43f6-b742-2c763375987c admin admin] PATH_INFO: `/v3/users/3dfe3bb587ec4adfab8245da46f9b24b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:08.399817 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1cf6f-894b-43f6-b742-2c763375987c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dfe3bb587ec4adfab8245da46f9b24b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:08.401784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1cf6f-894b-43f6-b742-2c763375987c admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:08.725101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290d2bc-ac64-4951-b7a9-544fbeefdd18 service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:08.725101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5290d2bc-ac64-4951-b7a9-544fbeefdd18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:08.750435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290d2bc-ac64-4951-b7a9-544fbeefdd18 service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:08.758162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d9bce-ba84-44c2-894a-34dcfe21297a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:08.760159 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d9bce-ba84-44c2-894a-34dcfe21297a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:08.760455 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d9bce-ba84-44c2-894a-34dcfe21297a admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:08.761205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d9bce-ba84-44c2-894a-34dcfe21297a admin admin] PATH_INFO: `/v3/projects/f47f6750e93744caa2562e14bedaaa29` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:08.763722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d9bce-ba84-44c2-894a-34dcfe21297a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f47f6750e93744caa2562e14bedaaa29)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:08.768549 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d9bce-ba84-44c2-894a-34dcfe21297a admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:08.872045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-24d1cf6f-894b-43f6-b742-2c763375987c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dfe3bb587ec4adfab8245da46f9b24b'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:08.890414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-24d1cf6f-894b-43f6-b742-2c763375987c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dfe3bb587ec4adfab8245da46f9b24b'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:08.891200 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-24d1cf6f-894b-43f6-b742-2c763375987c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dfe3bb587ec4adfab8245da46f9b24b'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:08.891862 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-24d1cf6f-894b-43f6-b742-2c763375987c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dfe3bb587ec4adfab8245da46f9b24b'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:08.907646 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-24d1cf6f-894b-43f6-b742-2c763375987c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dfe3bb587ec4adfab8245da46f9b24b'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:09.174569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d97d9bce-ba84-44c2-894a-34dcfe21297a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f47f6750e93744caa2562e14bedaaa29', 'request_id': 'req-d97d9bce-ba84-44c2-894a-34dcfe21297a'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:09.184018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d97d9bce-ba84-44c2-894a-34dcfe21297a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f47f6750e93744caa2562e14bedaaa29', 'request_id': 'req-d97d9bce-ba84-44c2-894a-34dcfe21297a'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:09.468745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1086/2174] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:26:08 2022] DELETE /identity/v3/users/3dfe3bb587ec4adfab8245da46f9b24b => generated 0 bytes in 1235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:26:09.471622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26788898-43e7-40d4-a993-7acf0302998e None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:09.471925 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26788898-43e7-40d4-a993-7acf0302998e None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:09.472827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26788898-43e7-40d4-a993-7acf0302998e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:09.515620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d97d9bce-ba84-44c2-894a-34dcfe21297a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f47f6750e93744caa2562e14bedaaa29', 'request_id': 'req-d97d9bce-ba84-44c2-894a-34dcfe21297a'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:09.529134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-26788898-43e7-40d4-a993-7acf0302998e None None] MFA Rules not processed for user `29fe5fb181e2452388d24fd0b7aa5e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:09.531870 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-26788898-43e7-40d4-a993-7acf0302998e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:09.534735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d97d9bce-ba84-44c2-894a-34dcfe21297a admin admin] The token cache is being invalidate because project f47f6750e93744caa2562e14bedaaa29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:26:09.535140 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d97d9bce-ba84-44c2-894a-34dcfe21297a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:09.537856 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-d97d9bce-ba84-44c2-894a-34dcfe21297a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:09.540096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1089/2175] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:26:08 2022] DELETE /identity/v3/projects/f47f6750e93744caa2562e14bedaaa29 => generated 0 bytes in 824 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:09.540741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d9bce-ba84-44c2-894a-34dcfe21297a admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:09.544430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d97d9bce-ba84-44c2-894a-34dcfe21297a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:09.628291 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1087/2176] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:09.629728 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fad415-cb7e-4fb1-b32b-ca5a8756e650 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:09.630023 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fad415-cb7e-4fb1-b32b-ca5a8756e650 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:09.630305 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fad415-cb7e-4fb1-b32b-ca5a8756e650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:09.679424 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d9bce-ba84-44c2-894a-34dcfe21297a admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:09.682220 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-84fad415-cb7e-4fb1-b32b-ca5a8756e650 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:09.683500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-84fad415-cb7e-4fb1-b32b-ca5a8756e650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:09.797520 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85bd7d0-0f88-45bf-8235-9588c6281815 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:09.800994 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85bd7d0-0f88-45bf-8235-9588c6281815 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:09.800994 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85bd7d0-0f88-45bf-8235-9588c6281815 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:09.800994 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85bd7d0-0f88-45bf-8235-9588c6281815 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:09.802696 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d85bd7d0-0f88-45bf-8235-9588c6281815 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:09.809713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1088/2177] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:09.822058 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59673e84-f846-44bd-ad7e-67902744190e None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:09.822366 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59673e84-f846-44bd-ad7e-67902744190e None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:09.822657 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59673e84-f846-44bd-ad7e-67902744190e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:09.870876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-59673e84-f846-44bd-ad7e-67902744190e None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:09.870876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-59673e84-f846-44bd-ad7e-67902744190e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:09.898947 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1089/2178] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:09.899737 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59673e84-f846-44bd-ad7e-67902744190e None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:09.904345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-59673e84-f846-44bd-ad7e-67902744190e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:09.908104 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85bd7d0-0f88-45bf-8235-9588c6281815 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:09.909599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85bd7d0-0f88-45bf-8235-9588c6281815 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:09.916053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:09.930046 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1090/2179] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:09 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:09.931435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2710fcd-0e81-42a8-a74f-5eed7622f508 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:09.931732 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2710fcd-0e81-42a8-a74f-5eed7622f508 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:09.932014 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2710fcd-0e81-42a8-a74f-5eed7622f508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:09.986573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-b2710fcd-0e81-42a8-a74f-5eed7622f508 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:09.989716 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-b2710fcd-0e81-42a8-a74f-5eed7622f508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:10.035704 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1091/2180] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:10.037313 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2710fcd-0e81-42a8-a74f-5eed7622f508 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:10.055845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59673e84-f846-44bd-ad7e-67902744190e None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:10.058013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2710fcd-0e81-42a8-a74f-5eed7622f508 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:10.068391 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc7563-f929-47b0-9037-9e1d78a36405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:10.070333 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc7563-f929-47b0-9037-9e1d78a36405 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:10.070849 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc7563-f929-47b0-9037-9e1d78a36405 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:10.071419 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc7563-f929-47b0-9037-9e1d78a36405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:10.073117 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f861ee-883c-45f9-a393-1cec39b20248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:10.073862 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f861ee-883c-45f9-a393-1cec39b20248 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:10.074151 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f861ee-883c-45f9-a393-1cec39b20248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:10.074511 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f861ee-883c-45f9-a393-1cec39b20248 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:10.075258 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f861ee-883c-45f9-a393-1cec39b20248 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:26:10.075518 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f861ee-883c-45f9-a393-1cec39b20248 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:10.077966 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f861ee-883c-45f9-a393-1cec39b20248 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:10.107489 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc7563-f929-47b0-9037-9e1d78a36405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:10.110849 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc7563-f929-47b0-9037-9e1d78a36405 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:10.119475 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1092/2181] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:26:10.126155 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f861ee-883c-45f9-a393-1cec39b20248 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:10.133067 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:10.144097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f861ee-883c-45f9-a393-1cec39b20248 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:10.152749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1090/2182] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:10.177511 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a09a97-0324-4dfe-b328-004518c717b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:10.178813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a09a97-0324-4dfe-b328-004518c717b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:10.179459 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a09a97-0324-4dfe-b328-004518c717b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:10.179966 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a09a97-0324-4dfe-b328-004518c717b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:10.180903 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a09a97-0324-4dfe-b328-004518c717b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:10.183305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a09a97-0324-4dfe-b328-004518c717b3 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:10.333012 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1093/2183] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:26:10 2022] POST /identity/v3/projects => generated 371 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:26:10.337399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc7563-f929-47b0-9037-9e1d78a36405 service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:10.356448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc7563-f929-47b0-9037-9e1d78a36405 service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:10.501347 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2e92b-162e-4594-b742-3cb1fa7dc481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:10.502132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2e92b-162e-4594-b742-3cb1fa7dc481 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:10.502332 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2e92b-162e-4594-b742-3cb1fa7dc481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:10.502612 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2e92b-162e-4594-b742-3cb1fa7dc481 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:10.503352 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d2e92b-162e-4594-b742-3cb1fa7dc481 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:10.505295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d2e92b-162e-4594-b742-3cb1fa7dc481 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:10.688363 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1091/2184] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:26:10 2022] POST /identity/v3/users => generated 286 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:26:10.694829 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a09a97-0324-4dfe-b328-004518c717b3 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:10.711384 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a09a97-0324-4dfe-b328-004518c717b3 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:10.721898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea1f5c-d30c-4897-9e85-a6f977b451ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:10.722772 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea1f5c-d30c-4897-9e85-a6f977b451ba admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:10.724368 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea1f5c-d30c-4897-9e85-a6f977b451ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:10.732097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea1f5c-d30c-4897-9e85-a6f977b451ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:10.732097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea1f5c-d30c-4897-9e85-a6f977b451ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:10.732097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea1f5c-d30c-4897-9e85-a6f977b451ba admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:10.745539 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1094/2185] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:26:10.755032 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2e92b-162e-4594-b742-3cb1fa7dc481 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:10.772806 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2e92b-162e-4594-b742-3cb1fa7dc481 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:10.781613 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223352d7-c99f-4c5e-8aaf-eec826c70a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:10.783441 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223352d7-c99f-4c5e-8aaf-eec826c70a63 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:10.784014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223352d7-c99f-4c5e-8aaf-eec826c70a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:10.784958 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223352d7-c99f-4c5e-8aaf-eec826c70a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:10.786945 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223352d7-c99f-4c5e-8aaf-eec826c70a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:10.791452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223352d7-c99f-4c5e-8aaf-eec826c70a63 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:10.814606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea1f5c-d30c-4897-9e85-a6f977b451ba admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:10.826123 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1092/2186] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:26:10.829197 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea1f5c-d30c-4897-9e85-a6f977b451ba admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:10.834247 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e19803-6a58-429b-92c4-fc6ffca1bd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:10.835424 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e19803-6a58-429b-92c4-fc6ffca1bd3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:10.835790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e19803-6a58-429b-92c4-fc6ffca1bd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:10.836138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e19803-6a58-429b-92c4-fc6ffca1bd3a admin admin] PATH_INFO: `/v3/projects/6a1a64d6d0f241739bb3dad4490182ed/users/484c9ed8db294be9aaeb2a7e50afabe0/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:10.867607 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e19803-6a58-429b-92c4-fc6ffca1bd3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a1a64d6d0f241739bb3dad4490182ed, user_id=484c9ed8db294be9aaeb2a7e50afabe0, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:10.870586 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e19803-6a58-429b-92c4-fc6ffca1bd3a admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:11.018299 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1095/2187] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:26:10 2022] PUT /identity/v3/projects/6a1a64d6d0f241739bb3dad4490182ed/users/484c9ed8db294be9aaeb2a7e50afabe0/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:26:11.026591 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431fd11-5e27-4bec-9a42-bc95a32283a8 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:11.026591 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431fd11-5e27-4bec-9a42-bc95a32283a8 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:11.026591 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431fd11-5e27-4bec-9a42-bc95a32283a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:11.056807 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-7431fd11-5e27-4bec-9a42-bc95a32283a8 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:11.064744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7431fd11-5e27-4bec-9a42-bc95a32283a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:11.148669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1093/2188] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:11.165486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e19803-6a58-429b-92c4-fc6ffca1bd3a admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:11.275224 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431fd11-5e27-4bec-9a42-bc95a32283a8 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:11.277116 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7431fd11-5e27-4bec-9a42-bc95a32283a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:11.292782 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e19803-6a58-429b-92c4-fc6ffca1bd3a admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:11.300768 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f32679-01f3-4e82-a47b-d76082feb150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:11.301751 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f32679-01f3-4e82-a47b-d76082feb150 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:11.303264 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f32679-01f3-4e82-a47b-d76082feb150 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:11.304503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f32679-01f3-4e82-a47b-d76082feb150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:11.308637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431fd11-5e27-4bec-9a42-bc95a32283a8 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:11.337748 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f338f77-fdc2-4393-b5bd-9789efdf8ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:11.337748 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f338f77-fdc2-4393-b5bd-9789efdf8ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:11.337748 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f338f77-fdc2-4393-b5bd-9789efdf8ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:11.337748 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f338f77-fdc2-4393-b5bd-9789efdf8ff0 admin admin] PATH_INFO: `/v3/projects/11d4f13492884e99a4ad6014bb470acc` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:11.351960 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f338f77-fdc2-4393-b5bd-9789efdf8ff0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11d4f13492884e99a4ad6014bb470acc)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:11.354275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f338f77-fdc2-4393-b5bd-9789efdf8ff0 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:11.378877 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f32679-01f3-4e82-a47b-d76082feb150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:11.380383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f32679-01f3-4e82-a47b-d76082feb150 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:11.398589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1096/2189] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:11.537022 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:11.567870 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-9f338f77-fdc2-4393-b5bd-9789efdf8ff0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11d4f13492884e99a4ad6014bb470acc', 'request_id': 'req-9f338f77-fdc2-4393-b5bd-9789efdf8ff0'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:11.578745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-9f338f77-fdc2-4393-b5bd-9789efdf8ff0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11d4f13492884e99a4ad6014bb470acc', 'request_id': 'req-9f338f77-fdc2-4393-b5bd-9789efdf8ff0'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:11.590576 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-9f338f77-fdc2-4393-b5bd-9789efdf8ff0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11d4f13492884e99a4ad6014bb470acc', 'request_id': 'req-9f338f77-fdc2-4393-b5bd-9789efdf8ff0'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:11.646573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:11.831503 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-9f338f77-fdc2-4393-b5bd-9789efdf8ff0 admin admin] The token cache is being invalidate because project 11d4f13492884e99a4ad6014bb470acc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:26:11.831503 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-9f338f77-fdc2-4393-b5bd-9789efdf8ff0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:11.831503 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-9f338f77-fdc2-4393-b5bd-9789efdf8ff0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:11.832613 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1094/2190] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:26:11 2022] DELETE /identity/v3/projects/11d4f13492884e99a4ad6014bb470acc => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:11.838099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1bc3c-1740-45ca-b1b2-2a932cb3486f None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:11.838394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1bc3c-1740-45ca-b1b2-2a932cb3486f None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:11.838679 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1bc3c-1740-45ca-b1b2-2a932cb3486f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:11.871123 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-afe1bc3c-1740-45ca-b1b2-2a932cb3486f None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:11.874270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-afe1bc3c-1740-45ca-b1b2-2a932cb3486f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:12.013931 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1097/2191] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:12.018775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad258edd-ff8b-4d0e-bb16-ac3bfab01114 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:12.018775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad258edd-ff8b-4d0e-bb16-ac3bfab01114 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:12.018775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad258edd-ff8b-4d0e-bb16-ac3bfab01114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:12.043774 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-ad258edd-ff8b-4d0e-bb16-ac3bfab01114 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:12.045112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-ad258edd-ff8b-4d0e-bb16-ac3bfab01114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:12.065149 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1095/2192] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:26:12.065995 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1bc3c-1740-45ca-b1b2-2a932cb3486f None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:12.078774 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1bc3c-1740-45ca-b1b2-2a932cb3486f None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:12.084979 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de365f-de38-4054-bcc3-6dc2bcf103e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:12.086154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de365f-de38-4054-bcc3-6dc2bcf103e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:12.089770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de365f-de38-4054-bcc3-6dc2bcf103e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:12.089770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de365f-de38-4054-bcc3-6dc2bcf103e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:12.089770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de365f-de38-4054-bcc3-6dc2bcf103e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:26:12.089770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de365f-de38-4054-bcc3-6dc2bcf103e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:12.090108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de365f-de38-4054-bcc3-6dc2bcf103e0 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:12.104106 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1098/2193] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:26:12.110972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad258edd-ff8b-4d0e-bb16-ac3bfab01114 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:12.120969 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad258edd-ff8b-4d0e-bb16-ac3bfab01114 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:12.134821 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef06bd-8493-409c-99f0-14eba6f356cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:12.135601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef06bd-8493-409c-99f0-14eba6f356cd admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:12.135897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef06bd-8493-409c-99f0-14eba6f356cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:12.136179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef06bd-8493-409c-99f0-14eba6f356cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:12.137027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef06bd-8493-409c-99f0-14eba6f356cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:12.139805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef06bd-8493-409c-99f0-14eba6f356cd admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:12.240672 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1096/2194] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:26:12 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:26:12.244385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de365f-de38-4054-bcc3-6dc2bcf103e0 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:12.256977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de365f-de38-4054-bcc3-6dc2bcf103e0 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:12.349004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b28304-f38e-43ea-b179-a4e6c2cd6c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:12.349910 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b28304-f38e-43ea-b179-a4e6c2cd6c3c admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:12.357022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b28304-f38e-43ea-b179-a4e6c2cd6c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:12.357463 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b28304-f38e-43ea-b179-a4e6c2cd6c3c admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:12.359533 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b28304-f38e-43ea-b179-a4e6c2cd6c3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:12.362055 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b28304-f38e-43ea-b179-a4e6c2cd6c3c admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:12.451990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1099/2195] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:26:12 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:26:12.456411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef06bd-8493-409c-99f0-14eba6f356cd admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:12.473264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef06bd-8493-409c-99f0-14eba6f356cd admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:12.478305 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac766b9-d8ad-4730-b0db-815b013f354f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:12.479018 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac766b9-d8ad-4730-b0db-815b013f354f admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:12.479312 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac766b9-d8ad-4730-b0db-815b013f354f admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:12.479604 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac766b9-d8ad-4730-b0db-815b013f354f admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:12.480348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac766b9-d8ad-4730-b0db-815b013f354f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:12.484688 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac766b9-d8ad-4730-b0db-815b013f354f admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:12.498954 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1097/2196] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:26:12.502406 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b28304-f38e-43ea-b179-a4e6c2cd6c3c admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:12.515048 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b28304-f38e-43ea-b179-a4e6c2cd6c3c admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:12.520270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d914b-a8c0-4097-97b3-525525c3359c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:12.521012 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d914b-a8c0-4097-97b3-525525c3359c admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:12.521364 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d914b-a8c0-4097-97b3-525525c3359c admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:12.521932 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d914b-a8c0-4097-97b3-525525c3359c admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:12.522780 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d914b-a8c0-4097-97b3-525525c3359c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:12.524601 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d914b-a8c0-4097-97b3-525525c3359c admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:12.549042 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1100/2197] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:26:12.553190 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac766b9-d8ad-4730-b0db-815b013f354f admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:12.564493 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac766b9-d8ad-4730-b0db-815b013f354f admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:12.576748 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63cb1d-e180-4da3-b5a6-d27ce61824b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:12.577170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63cb1d-e180-4da3-b5a6-d27ce61824b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:12.577461 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63cb1d-e180-4da3-b5a6-d27ce61824b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:12.577933 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63cb1d-e180-4da3-b5a6-d27ce61824b2 admin admin] PATH_INFO: `/v3/projects/abec122c2f0e4a50bfc0efe0e74b308f/users/f7a00c9f55f74697af2b6f465a52dfcc/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:12.602236 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63cb1d-e180-4da3-b5a6-d27ce61824b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abec122c2f0e4a50bfc0efe0e74b308f, user_id=f7a00c9f55f74697af2b6f465a52dfcc, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:12.608730 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63cb1d-e180-4da3-b5a6-d27ce61824b2 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:12.686941 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1098/2198] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:26:12 2022] PUT /identity/v3/projects/abec122c2f0e4a50bfc0efe0e74b308f/users/f7a00c9f55f74697af2b6f465a52dfcc/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:12.693414 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2dfdd9-09fe-4d66-9ed0-144c916c5f00 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:12.693867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2dfdd9-09fe-4d66-9ed0-144c916c5f00 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:12.694226 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2dfdd9-09fe-4d66-9ed0-144c916c5f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:12.723124 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:12.757741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-cb2dfdd9-09fe-4d66-9ed0-144c916c5f00 None None] MFA Rules not processed for user `f7a00c9f55f74697af2b6f465a52dfcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:12.757741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-cb2dfdd9-09fe-4d66-9ed0-144c916c5f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:12.938957 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1101/2199] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:12.956603 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63cb1d-e180-4da3-b5a6-d27ce61824b2 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:13.056021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63cb1d-e180-4da3-b5a6-d27ce61824b2 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:13.061544 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427b009-34e1-428c-b65a-eb87955a295c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:13.062313 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427b009-34e1-428c-b65a-eb87955a295c admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:13.062513 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427b009-34e1-428c-b65a-eb87955a295c admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:13.062795 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427b009-34e1-428c-b65a-eb87955a295c admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:13.063540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6427b009-34e1-428c-b65a-eb87955a295c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:13.065562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6427b009-34e1-428c-b65a-eb87955a295c admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:13.539067 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1099/2200] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:26:12 2022] POST /identity/v3/projects => generated 383 bytes in 584 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:26:13.543478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2dfdd9-09fe-4d66-9ed0-144c916c5f00 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:13.554804 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2dfdd9-09fe-4d66-9ed0-144c916c5f00 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:13.646830 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cffb0d-f023-4e90-a68f-e7677e482f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:13.647536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cffb0d-f023-4e90-a68f-e7677e482f25 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:13.648004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cffb0d-f023-4e90-a68f-e7677e482f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:13.648335 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cffb0d-f023-4e90-a68f-e7677e482f25 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:13.649054 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cffb0d-f023-4e90-a68f-e7677e482f25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:13.651039 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cffb0d-f023-4e90-a68f-e7677e482f25 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:13.697676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:13.855952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1102/2201] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:26:13 2022] POST /identity/v3/users => generated 292 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:26:13.857494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427b009-34e1-428c-b65a-eb87955a295c admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:13.869371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427b009-34e1-428c-b65a-eb87955a295c admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:13.878018 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdba209-dcb4-4422-afcc-d6a3f3071e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:13.878714 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdba209-dcb4-4422-afcc-d6a3f3071e89 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:13.879621 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdba209-dcb4-4422-afcc-d6a3f3071e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:13.879912 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdba209-dcb4-4422-afcc-d6a3f3071e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:13.880822 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdba209-dcb4-4422-afcc-d6a3f3071e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:13.882440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdba209-dcb4-4422-afcc-d6a3f3071e89 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:13.897569 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1100/2202] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:26:13.902067 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cffb0d-f023-4e90-a68f-e7677e482f25 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:13.922014 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cffb0d-f023-4e90-a68f-e7677e482f25 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:13.927276 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c88547-fdd4-46d8-bd71-e1d42354049f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:13.928009 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c88547-fdd4-46d8-bd71-e1d42354049f admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:13.928297 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c88547-fdd4-46d8-bd71-e1d42354049f admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:13.928712 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c88547-fdd4-46d8-bd71-e1d42354049f admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:13.929715 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c88547-fdd4-46d8-bd71-e1d42354049f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:13.934004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c88547-fdd4-46d8-bd71-e1d42354049f admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:13.944036 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1103/2203] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:26:13.948014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdba209-dcb4-4422-afcc-d6a3f3071e89 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:13.962972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdba209-dcb4-4422-afcc-d6a3f3071e89 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:13.969304 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee6dec-0933-440b-9e27-b38dc021aff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:13.970476 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee6dec-0933-440b-9e27-b38dc021aff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:13.970890 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee6dec-0933-440b-9e27-b38dc021aff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:13.971254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee6dec-0933-440b-9e27-b38dc021aff9 admin admin] PATH_INFO: `/v3/projects/bc47b6fc71bf4c2192459dea71872b16/users/9f7b1abf962e402a9237faabeac86ea5/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:14.002690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9ca6f-c9a9-442a-a335-1445fd60e372 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:14.002690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9ca6f-c9a9-442a-a335-1445fd60e372 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:14.002690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9ca6f-c9a9-442a-a335-1445fd60e372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:14.029208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee6dec-0933-440b-9e27-b38dc021aff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc47b6fc71bf4c2192459dea71872b16, user_id=9f7b1abf962e402a9237faabeac86ea5, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:14.036357 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee6dec-0933-440b-9e27-b38dc021aff9 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:14.057133 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-6bb9ca6f-c9a9-442a-a335-1445fd60e372 None None] MFA Rules not processed for user `484c9ed8db294be9aaeb2a7e50afabe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:14.058557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-6bb9ca6f-c9a9-442a-a335-1445fd60e372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:14.149801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1104/2204] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:26:14.173369 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9ca6f-c9a9-442a-a335-1445fd60e372 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:14.179451 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-6bb9ca6f-c9a9-442a-a335-1445fd60e372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:14.182783 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1101/2205] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:26:13 2022] PUT /identity/v3/projects/bc47b6fc71bf4c2192459dea71872b16/users/9f7b1abf962e402a9237faabeac86ea5/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:26:14.194873 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b67b9-a5cb-4499-95ed-bbb3c11b61b0 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:14.195298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b67b9-a5cb-4499-95ed-bbb3c11b61b0 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:14.195833 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b67b9-a5cb-4499-95ed-bbb3c11b61b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:14.255553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-9c3b67b9-a5cb-4499-95ed-bbb3c11b61b0 None None] MFA Rules not processed for user `9f7b1abf962e402a9237faabeac86ea5`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:14.258752 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-9c3b67b9-a5cb-4499-95ed-bbb3c11b61b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:14.321664 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9ca6f-c9a9-442a-a335-1445fd60e372 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:14.425043 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1102/2206] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:14.426368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b67b9-a5cb-4499-95ed-bbb3c11b61b0 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:14.439373 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ccf34-d157-48cb-bee6-0cdf3155a140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:14.446404 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ccf34-d157-48cb-bee6-0cdf3155a140 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:14.446404 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ccf34-d157-48cb-bee6-0cdf3155a140 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:14.446404 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ccf34-d157-48cb-bee6-0cdf3155a140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:14.523934 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ccf34-d157-48cb-bee6-0cdf3155a140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:14.523934 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ccf34-d157-48cb-bee6-0cdf3155a140 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:14.525712 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b67b9-a5cb-4499-95ed-bbb3c11b61b0 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:14.531668 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab28751-7aac-4544-96b6-7b8c468121b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:14.532450 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab28751-7aac-4544-96b6-7b8c468121b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:14.532817 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab28751-7aac-4544-96b6-7b8c468121b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:14.533178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab28751-7aac-4544-96b6-7b8c468121b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:14.534232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab28751-7aac-4544-96b6-7b8c468121b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:14.536386 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab28751-7aac-4544-96b6-7b8c468121b4 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:14.548159 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1105/2207] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:14.632439 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1103/2208] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:26:14 2022] POST /identity/v3/projects => generated 385 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:26:14.636310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ccf34-d157-48cb-bee6-0cdf3155a140 service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:14.652759 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ccf34-d157-48cb-bee6-0cdf3155a140 service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:14.681099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:14.729846 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25323ec2-451e-432b-a1e8-684e636b9c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:14.730567 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25323ec2-451e-432b-a1e8-684e636b9c0e admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:14.730888 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25323ec2-451e-432b-a1e8-684e636b9c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:14.731196 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25323ec2-451e-432b-a1e8-684e636b9c0e admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:14.731934 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25323ec2-451e-432b-a1e8-684e636b9c0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:14.734718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25323ec2-451e-432b-a1e8-684e636b9c0e admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:14.867637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1106/2209] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:26:14 2022] POST /identity/v3/users => generated 299 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:26:14.873480 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab28751-7aac-4544-96b6-7b8c468121b4 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:14.894551 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab28751-7aac-4544-96b6-7b8c468121b4 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:14.899566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0012279-fa05-4f7f-a814-bb053ee99e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:14.900371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0012279-fa05-4f7f-a814-bb053ee99e67 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:14.900739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0012279-fa05-4f7f-a814-bb053ee99e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:14.901088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0012279-fa05-4f7f-a814-bb053ee99e67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:14.901921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0012279-fa05-4f7f-a814-bb053ee99e67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:14.903663 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0012279-fa05-4f7f-a814-bb053ee99e67 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:14.920068 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1104/2210] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:26:14.924525 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25323ec2-451e-432b-a1e8-684e636b9c0e admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:14.935259 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25323ec2-451e-432b-a1e8-684e636b9c0e admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:14.943879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41161b2b-e470-464c-b4c5-b752facc3c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:14.944527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41161b2b-e470-464c-b4c5-b752facc3c74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:14.945031 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41161b2b-e470-464c-b4c5-b752facc3c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:14.945386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41161b2b-e470-464c-b4c5-b752facc3c74 admin admin] PATH_INFO: `/v3/projects/000776dc041d460e9f482155fb819094/users/a2c1e681c40f406085793ea5d75163c1/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:14.967725 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41161b2b-e470-464c-b4c5-b752facc3c74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=000776dc041d460e9f482155fb819094, user_id=a2c1e681c40f406085793ea5d75163c1, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:14.986328 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41161b2b-e470-464c-b4c5-b752facc3c74 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:15.101888 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:15.110426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1107/2211] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:26:14 2022] PUT /identity/v3/projects/000776dc041d460e9f482155fb819094/users/a2c1e681c40f406085793ea5d75163c1/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:15.114369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0012279-fa05-4f7f-a814-bb053ee99e67 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:15.129311 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0012279-fa05-4f7f-a814-bb053ee99e67 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:15.205255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d6d88-5eb4-46d1-82ba-7d858fb81a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:15.206657 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d6d88-5eb4-46d1-82ba-7d858fb81a41 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:15.207238 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d6d88-5eb4-46d1-82ba-7d858fb81a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:15.207904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d6d88-5eb4-46d1-82ba-7d858fb81a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:15.208962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d6d88-5eb4-46d1-82ba-7d858fb81a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:15.210902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d6d88-5eb4-46d1-82ba-7d858fb81a41 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:15.223774 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1105/2212] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:26:15.228078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41161b2b-e470-464c-b4c5-b752facc3c74 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:15.241894 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41161b2b-e470-464c-b4c5-b752facc3c74 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:15.247271 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1caac7-b75e-4423-91b0-b7228ab9a8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:15.248390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1caac7-b75e-4423-91b0-b7228ab9a8dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:15.248779 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1caac7-b75e-4423-91b0-b7228ab9a8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:15.249151 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1caac7-b75e-4423-91b0-b7228ab9a8dc admin admin] PATH_INFO: `/v3/projects/000776dc041d460e9f482155fb819094/users/a2c1e681c40f406085793ea5d75163c1/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:15.251312 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1caac7-b75e-4423-91b0-b7228ab9a8dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=000776dc041d460e9f482155fb819094, user_id=a2c1e681c40f406085793ea5d75163c1, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:15.254113 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1caac7-b75e-4423-91b0-b7228ab9a8dc admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:15.270190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1108/2213] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:26:15 2022] PUT /identity/v3/projects/000776dc041d460e9f482155fb819094/users/a2c1e681c40f406085793ea5d75163c1/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:15.281629 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fab57c-a9d9-4e19-b3b8-2ad07194f435 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:15.281991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fab57c-a9d9-4e19-b3b8-2ad07194f435 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:15.282383 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fab57c-a9d9-4e19-b3b8-2ad07194f435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:15.324090 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-b7fab57c-a9d9-4e19-b3b8-2ad07194f435 None None] MFA Rules not processed for user `a2c1e681c40f406085793ea5d75163c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:15.325897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b7fab57c-a9d9-4e19-b3b8-2ad07194f435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:15.436087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:15.446979 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1106/2214] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:15.471117 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1caac7-b75e-4423-91b0-b7228ab9a8dc admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:15.626372 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1caac7-b75e-4423-91b0-b7228ab9a8dc admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:15.634219 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687c5fd-32b6-4eb4-ba5e-547778c20a7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:15.636640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687c5fd-32b6-4eb4-ba5e-547778c20a7e service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:15.640750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687c5fd-32b6-4eb4-ba5e-547778c20a7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:15.640750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687c5fd-32b6-4eb4-ba5e-547778c20a7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:15.719927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687c5fd-32b6-4eb4-ba5e-547778c20a7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:15.721501 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687c5fd-32b6-4eb4-ba5e-547778c20a7e service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:15.732723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1109/2215] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:15.903481 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:16.583400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:16.753934 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:17.656150 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8e082-faeb-4713-85b3-7b15c4646920 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:17.656686 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8e082-faeb-4713-85b3-7b15c4646920 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:17.656686 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8e082-faeb-4713-85b3-7b15c4646920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:17.682862 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-11b8e082-faeb-4713-85b3-7b15c4646920 None None] MFA Rules not processed for user `484c9ed8db294be9aaeb2a7e50afabe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:17.684111 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-11b8e082-faeb-4713-85b3-7b15c4646920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:17.752815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1107/2216] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:18.561688 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:18.658362 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687c5fd-32b6-4eb4-ba5e-547778c20a7e service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:18.698005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687c5fd-32b6-4eb4-ba5e-547778c20a7e service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:18.720037 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db2d4d-e9a0-4740-b979-7049aa975091 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:18.723255 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db2d4d-e9a0-4740-b979-7049aa975091 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:18.724421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db2d4d-e9a0-4740-b979-7049aa975091 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:18.725142 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db2d4d-e9a0-4740-b979-7049aa975091 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:18.823631 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:18.870861 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8e082-faeb-4713-85b3-7b15c4646920 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:18.872379 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-11b8e082-faeb-4713-85b3-7b15c4646920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:18.877853 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db2d4d-e9a0-4740-b979-7049aa975091 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:18.879443 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db2d4d-e9a0-4740-b979-7049aa975091 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:18.889918 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1110/2217] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:19.005702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8e082-faeb-4713-85b3-7b15c4646920 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:19.012907 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251702c1-ca49-4a2f-a9e6-997d75021d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:19.016186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251702c1-ca49-4a2f-a9e6-997d75021d63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:19.016186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251702c1-ca49-4a2f-a9e6-997d75021d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:19.016186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251702c1-ca49-4a2f-a9e6-997d75021d63 admin admin] PATH_INFO: `/v3/users/69db77e4b32847249a6289fc48e75bef` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:19.017623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251702c1-ca49-4a2f-a9e6-997d75021d63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69db77e4b32847249a6289fc48e75bef)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:19.021627 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251702c1-ca49-4a2f-a9e6-997d75021d63 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:19.417935 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-251702c1-ca49-4a2f-a9e6-997d75021d63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69db77e4b32847249a6289fc48e75bef'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:19.424719 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-251702c1-ca49-4a2f-a9e6-997d75021d63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69db77e4b32847249a6289fc48e75bef'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:19.425445 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-251702c1-ca49-4a2f-a9e6-997d75021d63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69db77e4b32847249a6289fc48e75bef'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:19.426013 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-251702c1-ca49-4a2f-a9e6-997d75021d63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69db77e4b32847249a6289fc48e75bef'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:19.438310 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-251702c1-ca49-4a2f-a9e6-997d75021d63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69db77e4b32847249a6289fc48e75bef'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:19.695937 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1108/2218] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:26:18 2022] DELETE /identity/v3/users/69db77e4b32847249a6289fc48e75bef => generated 0 bytes in 826 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:19.700598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99728d-51a4-4a60-b43b-ba9c9da2c475 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:19.701003 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99728d-51a4-4a60-b43b-ba9c9da2c475 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:19.701915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99728d-51a4-4a60-b43b-ba9c9da2c475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:19.734026 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-5d99728d-51a4-4a60-b43b-ba9c9da2c475 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:19.735603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5d99728d-51a4-4a60-b43b-ba9c9da2c475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:19.796129 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1111/2219] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:19.806216 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251702c1-ca49-4a2f-a9e6-997d75021d63 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:19.807741 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-251702c1-ca49-4a2f-a9e6-997d75021d63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:19.907429 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251702c1-ca49-4a2f-a9e6-997d75021d63 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:19.911510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc468f91-627c-40a9-a6c2-03d6634b9592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:19.912216 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc468f91-627c-40a9-a6c2-03d6634b9592 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:19.912511 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc468f91-627c-40a9-a6c2-03d6634b9592 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:19.928873 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc468f91-627c-40a9-a6c2-03d6634b9592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:19.928873 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc468f91-627c-40a9-a6c2-03d6634b9592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:19.928873 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc468f91-627c-40a9-a6c2-03d6634b9592 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:19.935218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1109/2220] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:20.072721 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:20.505236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99728d-51a4-4a60-b43b-ba9c9da2c475 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:20.507443 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5d99728d-51a4-4a60-b43b-ba9c9da2c475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:20.531637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99728d-51a4-4a60-b43b-ba9c9da2c475 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:20.536892 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c74f85-4217-4c5e-8d81-1173ae09a6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:20.543052 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c74f85-4217-4c5e-8d81-1173ae09a6be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:20.543052 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c74f85-4217-4c5e-8d81-1173ae09a6be admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:20.543052 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c74f85-4217-4c5e-8d81-1173ae09a6be admin admin] PATH_INFO: `/v3/projects/a3469e42573a465d9520c90c06eb78d0` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:20.543052 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c74f85-4217-4c5e-8d81-1173ae09a6be admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3469e42573a465d9520c90c06eb78d0)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:20.543052 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c74f85-4217-4c5e-8d81-1173ae09a6be admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:20.722371 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-b0c74f85-4217-4c5e-8d81-1173ae09a6be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3469e42573a465d9520c90c06eb78d0', 'request_id': 'req-b0c74f85-4217-4c5e-8d81-1173ae09a6be'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:20.728707 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-b0c74f85-4217-4c5e-8d81-1173ae09a6be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3469e42573a465d9520c90c06eb78d0', 'request_id': 'req-b0c74f85-4217-4c5e-8d81-1173ae09a6be'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:20.851129 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-b0c74f85-4217-4c5e-8d81-1173ae09a6be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3469e42573a465d9520c90c06eb78d0', 'request_id': 'req-b0c74f85-4217-4c5e-8d81-1173ae09a6be'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:20.864249 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-b0c74f85-4217-4c5e-8d81-1173ae09a6be admin admin] The token cache is being invalidate because project a3469e42573a465d9520c90c06eb78d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:26:20.864545 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-b0c74f85-4217-4c5e-8d81-1173ae09a6be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:20.865200 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-b0c74f85-4217-4c5e-8d81-1173ae09a6be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:20.866801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1112/2221] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:26:20 2022] DELETE /identity/v3/projects/a3469e42573a465d9520c90c06eb78d0 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:20.870792 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc468f91-627c-40a9-a6c2-03d6634b9592 service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:20.881105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc468f91-627c-40a9-a6c2-03d6634b9592 service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:20.988037 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d5420-24ff-4538-a1ac-c5a1a7e4574e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:20.989069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d5420-24ff-4538-a1ac-c5a1a7e4574e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:20.989427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d5420-24ff-4538-a1ac-c5a1a7e4574e admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:20.989804 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d5420-24ff-4538-a1ac-c5a1a7e4574e admin admin] PATH_INFO: `/v3/users/a6d9aadce6e1409aa7b9e6cbcf5444ef` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:20.991182 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d5420-24ff-4538-a1ac-c5a1a7e4574e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6d9aadce6e1409aa7b9e6cbcf5444ef)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:20.993465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d5420-24ff-4538-a1ac-c5a1a7e4574e admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:21.107020 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c74f85-4217-4c5e-8d81-1173ae09a6be admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:21.109578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-b0c74f85-4217-4c5e-8d81-1173ae09a6be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:21.202316 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c74f85-4217-4c5e-8d81-1173ae09a6be admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:21.211693 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ef6b5-8c7d-4439-83af-fee5d4569307 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:21.212674 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ef6b5-8c7d-4439-83af-fee5d4569307 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:21.213173 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ef6b5-8c7d-4439-83af-fee5d4569307 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:21.213672 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ef6b5-8c7d-4439-83af-fee5d4569307 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:21.216522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5d9ef6b5-8c7d-4439-83af-fee5d4569307 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:21.225119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:21.287553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-619d5420-24ff-4538-a1ac-c5a1a7e4574e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6d9aadce6e1409aa7b9e6cbcf5444ef'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:21.297054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-619d5420-24ff-4538-a1ac-c5a1a7e4574e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6d9aadce6e1409aa7b9e6cbcf5444ef'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:21.298009 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-619d5420-24ff-4538-a1ac-c5a1a7e4574e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6d9aadce6e1409aa7b9e6cbcf5444ef'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:21.299018 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-619d5420-24ff-4538-a1ac-c5a1a7e4574e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6d9aadce6e1409aa7b9e6cbcf5444ef'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:21.303151 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ef6b5-8c7d-4439-83af-fee5d4569307 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:21.304654 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ef6b5-8c7d-4439-83af-fee5d4569307 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:21.314648 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-619d5420-24ff-4538-a1ac-c5a1a7e4574e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6d9aadce6e1409aa7b9e6cbcf5444ef'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:21.316995 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1113/2222] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:26:21.394148 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1110/2223] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:26:20 2022] DELETE /identity/v3/users/a6d9aadce6e1409aa7b9e6cbcf5444ef => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:26:22.304552 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ef6b5-8c7d-4439-83af-fee5d4569307 service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:22.306043 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5d9ef6b5-8c7d-4439-83af-fee5d4569307 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:22.320383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ef6b5-8c7d-4439-83af-fee5d4569307 service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:22.377875 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f2c97-ae7b-4a7a-bc31-be8b02976fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:22.378885 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f2c97-ae7b-4a7a-bc31-be8b02976fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:22.379215 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f2c97-ae7b-4a7a-bc31-be8b02976fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:22.379492 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f2c97-ae7b-4a7a-bc31-be8b02976fec admin admin] PATH_INFO: `/v3/projects/f38a92d89edd40a9bf2c9f7a3101f618` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:22.380680 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f2c97-ae7b-4a7a-bc31-be8b02976fec admin admin] RBAC: Authorizing `identity:delete_project(project_id=f38a92d89edd40a9bf2c9f7a3101f618)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:22.382593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f2c97-ae7b-4a7a-bc31-be8b02976fec admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:22.539777 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-504f2c97-ae7b-4a7a-bc31-be8b02976fec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f38a92d89edd40a9bf2c9f7a3101f618', 'request_id': 'req-504f2c97-ae7b-4a7a-bc31-be8b02976fec'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:22.547094 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-504f2c97-ae7b-4a7a-bc31-be8b02976fec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f38a92d89edd40a9bf2c9f7a3101f618', 'request_id': 'req-504f2c97-ae7b-4a7a-bc31-be8b02976fec'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:22.633940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-504f2c97-ae7b-4a7a-bc31-be8b02976fec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f38a92d89edd40a9bf2c9f7a3101f618', 'request_id': 'req-504f2c97-ae7b-4a7a-bc31-be8b02976fec'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:22.645583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-504f2c97-ae7b-4a7a-bc31-be8b02976fec admin admin] The token cache is being invalidate because project f38a92d89edd40a9bf2c9f7a3101f618 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:26:22.645876 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-504f2c97-ae7b-4a7a-bc31-be8b02976fec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:22.646514 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-504f2c97-ae7b-4a7a-bc31-be8b02976fec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:22.648237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1114/2224] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:26:22 2022] DELETE /identity/v3/projects/f38a92d89edd40a9bf2c9f7a3101f618 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:22.653090 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d62ec-248b-4bd9-ad12-646c05d80880 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:22.653582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d62ec-248b-4bd9-ad12-646c05d80880 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:22.653683 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d62ec-248b-4bd9-ad12-646c05d80880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:22.672510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:22.694035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-362d62ec-248b-4bd9-ad12-646c05d80880 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:22.695375 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-362d62ec-248b-4bd9-ad12-646c05d80880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:22.758458 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1111/2225] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:22.766263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90236216-3da9-430d-a76d-975601ab253c None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:22.766420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90236216-3da9-430d-a76d-975601ab253c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:22.766701 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90236216-3da9-430d-a76d-975601ab253c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:22.784742 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-90236216-3da9-430d-a76d-975601ab253c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:22.785966 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-90236216-3da9-430d-a76d-975601ab253c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:22.794328 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1115/2226] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:22.798879 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d62ec-248b-4bd9-ad12-646c05d80880 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:22.809770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d62ec-248b-4bd9-ad12-646c05d80880 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:22.816005 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5c3b0-2ba2-4e31-a000-bd5e436fb58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:22.816834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5c3b0-2ba2-4e31-a000-bd5e436fb58a admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:22.816958 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5c3b0-2ba2-4e31-a000-bd5e436fb58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:22.817722 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5c3b0-2ba2-4e31-a000-bd5e436fb58a admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:22.817999 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5c3b0-2ba2-4e31-a000-bd5e436fb58a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:26:22.818400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5c3b0-2ba2-4e31-a000-bd5e436fb58a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:22.819946 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5c3b0-2ba2-4e31-a000-bd5e436fb58a admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:22.835130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1112/2227] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:26:22.838104 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90236216-3da9-430d-a76d-975601ab253c None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:22.852126 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90236216-3da9-430d-a76d-975601ab253c None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:22.862758 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec63053-c641-4bb9-9f28-3dcc7c27e532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:22.863787 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec63053-c641-4bb9-9f28-3dcc7c27e532 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:22.864332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec63053-c641-4bb9-9f28-3dcc7c27e532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:22.865293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec63053-c641-4bb9-9f28-3dcc7c27e532 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:22.866575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec63053-c641-4bb9-9f28-3dcc7c27e532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:22.878472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec63053-c641-4bb9-9f28-3dcc7c27e532 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:23.027553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1116/2228] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:26:22 2022] POST /identity/v3/projects => generated 387 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:26:23.029394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5c3b0-2ba2-4e31-a000-bd5e436fb58a admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:23.041054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5c3b0-2ba2-4e31-a000-bd5e436fb58a admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:23.112239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd045d-3543-4dce-931f-82f0726abe30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:23.113182 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd045d-3543-4dce-931f-82f0726abe30 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:23.113483 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd045d-3543-4dce-931f-82f0726abe30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:23.113766 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd045d-3543-4dce-931f-82f0726abe30 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:23.114482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd045d-3543-4dce-931f-82f0726abe30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:23.116466 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd045d-3543-4dce-931f-82f0726abe30 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:23.251854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1113/2229] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:26:23 2022] POST /identity/v3/users => generated 294 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:26:23.256155 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec63053-c641-4bb9-9f28-3dcc7c27e532 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:23.279773 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec63053-c641-4bb9-9f28-3dcc7c27e532 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:23.285236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177728a-3f34-452e-81cd-f1ef35347542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:23.286042 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177728a-3f34-452e-81cd-f1ef35347542 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:23.286338 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177728a-3f34-452e-81cd-f1ef35347542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:23.286644 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177728a-3f34-452e-81cd-f1ef35347542 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:23.287860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4177728a-3f34-452e-81cd-f1ef35347542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:23.289523 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4177728a-3f34-452e-81cd-f1ef35347542 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:23.301094 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1117/2230] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:26:23.305143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd045d-3543-4dce-931f-82f0726abe30 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:23.319213 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd045d-3543-4dce-931f-82f0726abe30 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:23.324137 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41534de8-4585-4523-b5c3-a38e74313558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:23.324927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41534de8-4585-4523-b5c3-a38e74313558 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:23.325426 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41534de8-4585-4523-b5c3-a38e74313558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:23.325623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41534de8-4585-4523-b5c3-a38e74313558 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:23.327099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41534de8-4585-4523-b5c3-a38e74313558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:23.328793 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41534de8-4585-4523-b5c3-a38e74313558 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:23.342675 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1114/2231] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:26:23.346980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177728a-3f34-452e-81cd-f1ef35347542 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:23.361770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177728a-3f34-452e-81cd-f1ef35347542 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:23.366685 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c2e68-8897-46a2-9541-dd2b5f8418df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:23.367718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c2e68-8897-46a2-9541-dd2b5f8418df admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:23.368007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c2e68-8897-46a2-9541-dd2b5f8418df admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:23.368288 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c2e68-8897-46a2-9541-dd2b5f8418df admin admin] PATH_INFO: `/v3/projects/389e3c05babf477fad849535063df2ce/users/15f0192fe47f4eb6bd07ff2279281630/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:23.384584 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c2e68-8897-46a2-9541-dd2b5f8418df admin admin] RBAC: Authorizing `identity:create_grant(project_id=389e3c05babf477fad849535063df2ce, user_id=15f0192fe47f4eb6bd07ff2279281630, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:23.387588 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c2e68-8897-46a2-9541-dd2b5f8418df admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:23.468978 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1118/2232] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:26:23 2022] PUT /identity/v3/projects/389e3c05babf477fad849535063df2ce/users/15f0192fe47f4eb6bd07ff2279281630/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:23.479294 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d677d-ca06-4901-95a6-0d354e889fa7 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:23.479588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d677d-ca06-4901-95a6-0d354e889fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:23.479869 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d677d-ca06-4901-95a6-0d354e889fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:23.510313 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-480d677d-ca06-4901-95a6-0d354e889fa7 None None] MFA Rules not processed for user `15f0192fe47f4eb6bd07ff2279281630`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:23.510313 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-480d677d-ca06-4901-95a6-0d354e889fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:23.573521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1115/2233] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:23.578106 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c2e68-8897-46a2-9541-dd2b5f8418df admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:23.641647 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c2e68-8897-46a2-9541-dd2b5f8418df admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:23.646382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22703714-6d0b-4a23-bed1-d5bb21bfd5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:23.647085 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22703714-6d0b-4a23-bed1-d5bb21bfd5ca admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:23.647392 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22703714-6d0b-4a23-bed1-d5bb21bfd5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:23.647656 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22703714-6d0b-4a23-bed1-d5bb21bfd5ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:23.648493 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22703714-6d0b-4a23-bed1-d5bb21bfd5ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:23.650360 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22703714-6d0b-4a23-bed1-d5bb21bfd5ca admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:23.814067 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1119/2234] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:26:23 2022] POST /identity/v3/projects => generated 389 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:26:23.815382 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d677d-ca06-4901-95a6-0d354e889fa7 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:23.826696 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d677d-ca06-4901-95a6-0d354e889fa7 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:23.884010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63ebeb-6c6e-4403-a903-5bf58d333082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:23.884703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63ebeb-6c6e-4403-a903-5bf58d333082 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:23.885070 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63ebeb-6c6e-4403-a903-5bf58d333082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:23.885384 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63ebeb-6c6e-4403-a903-5bf58d333082 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:23.886121 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63ebeb-6c6e-4403-a903-5bf58d333082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:23.888361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63ebeb-6c6e-4403-a903-5bf58d333082 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:23.948848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:24.027826 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1116/2235] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:26:23 2022] POST /identity/v3/users => generated 301 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:26:24.029147 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22703714-6d0b-4a23-bed1-d5bb21bfd5ca admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:24.040764 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22703714-6d0b-4a23-bed1-d5bb21bfd5ca admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:24.046968 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa8bee-3e95-4d17-8aca-31f633d48898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:24.050489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa8bee-3e95-4d17-8aca-31f633d48898 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:24.050489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa8bee-3e95-4d17-8aca-31f633d48898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:24.050489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa8bee-3e95-4d17-8aca-31f633d48898 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:24.050489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa8bee-3e95-4d17-8aca-31f633d48898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:24.050951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa8bee-3e95-4d17-8aca-31f633d48898 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:24.063968 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1120/2236] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:26:24.068453 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63ebeb-6c6e-4403-a903-5bf58d333082 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:24.087313 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63ebeb-6c6e-4403-a903-5bf58d333082 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:24.092894 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5e6e5-7996-42bf-a947-9993ef691569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:24.093794 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5e6e5-7996-42bf-a947-9993ef691569 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:24.094089 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5e6e5-7996-42bf-a947-9993ef691569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:24.094369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5e6e5-7996-42bf-a947-9993ef691569 admin admin] PATH_INFO: `/v3/projects/b9322efcce0f4b828c34c9de5d0739d8/users/5abab218b32a43c9996f8791009415a7/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:24.113909 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5e6e5-7996-42bf-a947-9993ef691569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9322efcce0f4b828c34c9de5d0739d8, user_id=5abab218b32a43c9996f8791009415a7, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:24.117342 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5e6e5-7996-42bf-a947-9993ef691569 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:24.219598 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1117/2237] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:26:24 2022] PUT /identity/v3/projects/b9322efcce0f4b828c34c9de5d0739d8/users/5abab218b32a43c9996f8791009415a7/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:24.223458 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa8bee-3e95-4d17-8aca-31f633d48898 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:24.241657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa8bee-3e95-4d17-8aca-31f633d48898 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:24.321576 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13d476-b9d2-41bb-95d4-c5b15870b5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:24.321576 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13d476-b9d2-41bb-95d4-c5b15870b5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:24.321887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13d476-b9d2-41bb-95d4-c5b15870b5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:24.322009 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13d476-b9d2-41bb-95d4-c5b15870b5a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:24.323450 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d13d476-b9d2-41bb-95d4-c5b15870b5a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:24.325324 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d13d476-b9d2-41bb-95d4-c5b15870b5a8 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:24.348317 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1121/2238] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:26:24.348861 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5e6e5-7996-42bf-a947-9993ef691569 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:24.356037 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5e6e5-7996-42bf-a947-9993ef691569 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:24.364579 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee2c62-0aed-49fa-94f0-de1dd89df2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:24.367314 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee2c62-0aed-49fa-94f0-de1dd89df2b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:24.367613 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee2c62-0aed-49fa-94f0-de1dd89df2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:24.367902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee2c62-0aed-49fa-94f0-de1dd89df2b5 admin admin] PATH_INFO: `/v3/projects/b9322efcce0f4b828c34c9de5d0739d8/users/5abab218b32a43c9996f8791009415a7/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:24.370041 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee2c62-0aed-49fa-94f0-de1dd89df2b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9322efcce0f4b828c34c9de5d0739d8, user_id=5abab218b32a43c9996f8791009415a7, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:24.373474 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee2c62-0aed-49fa-94f0-de1dd89df2b5 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:24.389312 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1118/2239] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:26:24 2022] PUT /identity/v3/projects/b9322efcce0f4b828c34c9de5d0739d8/users/5abab218b32a43c9996f8791009415a7/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:24.409512 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed99133-d1a3-43d9-9a9d-4e80e28f66fc None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:24.409816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed99133-d1a3-43d9-9a9d-4e80e28f66fc None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:24.410102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed99133-d1a3-43d9-9a9d-4e80e28f66fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:24.472190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-6ed99133-d1a3-43d9-9a9d-4e80e28f66fc None None] MFA Rules not processed for user `5abab218b32a43c9996f8791009415a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:24.473691 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-6ed99133-d1a3-43d9-9a9d-4e80e28f66fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:24.552144 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1122/2240] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:24 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:24.674699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee2c62-0aed-49fa-94f0-de1dd89df2b5 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:24.676391 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-77ee2c62-0aed-49fa-94f0-de1dd89df2b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:24.754488 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee2c62-0aed-49fa-94f0-de1dd89df2b5 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:24.758582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62779cf1-4938-44ca-84a9-239eb2faf743 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:24.759344 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62779cf1-4938-44ca-84a9-239eb2faf743 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:24.759635 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62779cf1-4938-44ca-84a9-239eb2faf743 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:24.759915 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62779cf1-4938-44ca-84a9-239eb2faf743 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:24.771957 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62779cf1-4938-44ca-84a9-239eb2faf743 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:24.773515 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62779cf1-4938-44ca-84a9-239eb2faf743 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:24.783716 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1119/2241] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:24 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:25.039451 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed99133-d1a3-43d9-9a9d-4e80e28f66fc None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:25.113524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed99133-d1a3-43d9-9a9d-4e80e28f66fc None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:25.129308 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a66fd1-cc0b-447e-9ff7-96d8e1397bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:25.130325 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a66fd1-cc0b-447e-9ff7-96d8e1397bd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:25.130893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a66fd1-cc0b-447e-9ff7-96d8e1397bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:25.131651 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a66fd1-cc0b-447e-9ff7-96d8e1397bd5 admin admin] PATH_INFO: `/v3/users/15f0192fe47f4eb6bd07ff2279281630` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:25.133151 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a66fd1-cc0b-447e-9ff7-96d8e1397bd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15f0192fe47f4eb6bd07ff2279281630)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:25.135212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a66fd1-cc0b-447e-9ff7-96d8e1397bd5 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:25.291643 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:25.423023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-47a66fd1-cc0b-447e-9ff7-96d8e1397bd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15f0192fe47f4eb6bd07ff2279281630'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:25.430244 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-47a66fd1-cc0b-447e-9ff7-96d8e1397bd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15f0192fe47f4eb6bd07ff2279281630'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:25.431060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-47a66fd1-cc0b-447e-9ff7-96d8e1397bd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15f0192fe47f4eb6bd07ff2279281630'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:25.431975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-47a66fd1-cc0b-447e-9ff7-96d8e1397bd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15f0192fe47f4eb6bd07ff2279281630'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:25.446335 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-47a66fd1-cc0b-447e-9ff7-96d8e1397bd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15f0192fe47f4eb6bd07ff2279281630'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:25.605265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1123/2242] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:26:25 2022] DELETE /identity/v3/users/15f0192fe47f4eb6bd07ff2279281630 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:25.610982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd96be-ef72-4c6a-adb0-bdff636a932f None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:25.611366 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd96be-ef72-4c6a-adb0-bdff636a932f None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:25.611889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd96be-ef72-4c6a-adb0-bdff636a932f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:25.630984 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-e8fd96be-ef72-4c6a-adb0-bdff636a932f None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:25.632407 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e8fd96be-ef72-4c6a-adb0-bdff636a932f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:25.703379 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1120/2243] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:25.714683 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a66fd1-cc0b-447e-9ff7-96d8e1397bd5 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:25.716357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-47a66fd1-cc0b-447e-9ff7-96d8e1397bd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:25.792242 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a66fd1-cc0b-447e-9ff7-96d8e1397bd5 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:25.796381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759c1f2-231d-46cf-80e8-576e3c08421c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:25.797124 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759c1f2-231d-46cf-80e8-576e3c08421c service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:25.797415 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759c1f2-231d-46cf-80e8-576e3c08421c service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:25.797801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759c1f2-231d-46cf-80e8-576e3c08421c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:25.812154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b759c1f2-231d-46cf-80e8-576e3c08421c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:25.812154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b759c1f2-231d-46cf-80e8-576e3c08421c service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:25.823088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1124/2244] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:26.635806 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd96be-ef72-4c6a-adb0-bdff636a932f None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:26.637846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e8fd96be-ef72-4c6a-adb0-bdff636a932f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:26.662721 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd96be-ef72-4c6a-adb0-bdff636a932f None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:26.681564 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984854d3-93e6-4540-8ce0-b9a16f7d2e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:26.681564 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984854d3-93e6-4540-8ce0-b9a16f7d2e48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:26.681564 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984854d3-93e6-4540-8ce0-b9a16f7d2e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:26.682566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984854d3-93e6-4540-8ce0-b9a16f7d2e48 admin admin] PATH_INFO: `/v3/projects/389e3c05babf477fad849535063df2ce` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:26.689708 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984854d3-93e6-4540-8ce0-b9a16f7d2e48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=389e3c05babf477fad849535063df2ce)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:26.692185 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984854d3-93e6-4540-8ce0-b9a16f7d2e48 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:26.764570 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759c1f2-231d-46cf-80e8-576e3c08421c service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:26.767306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-b759c1f2-231d-46cf-80e8-576e3c08421c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:26.785983 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759c1f2-231d-46cf-80e8-576e3c08421c service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:26.796349 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e6056-90a3-4ac7-a9d8-7c8ced6f4852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:26.797952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e6056-90a3-4ac7-a9d8-7c8ced6f4852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:26.798461 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e6056-90a3-4ac7-a9d8-7c8ced6f4852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:26.798952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e6056-90a3-4ac7-a9d8-7c8ced6f4852 admin admin] PATH_INFO: `/v3/users/f7a00c9f55f74697af2b6f465a52dfcc` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:26.800426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e6056-90a3-4ac7-a9d8-7c8ced6f4852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7a00c9f55f74697af2b6f465a52dfcc)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:26.802555 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e6056-90a3-4ac7-a9d8-7c8ced6f4852 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:27.162278 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-984854d3-93e6-4540-8ce0-b9a16f7d2e48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '389e3c05babf477fad849535063df2ce', 'request_id': 'req-984854d3-93e6-4540-8ce0-b9a16f7d2e48'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:27.163096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-b57e6056-90a3-4ac7-a9d8-7c8ced6f4852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7a00c9f55f74697af2b6f465a52dfcc'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:27.169696 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-984854d3-93e6-4540-8ce0-b9a16f7d2e48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '389e3c05babf477fad849535063df2ce', 'request_id': 'req-984854d3-93e6-4540-8ce0-b9a16f7d2e48'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:27.177915 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-984854d3-93e6-4540-8ce0-b9a16f7d2e48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '389e3c05babf477fad849535063df2ce', 'request_id': 'req-984854d3-93e6-4540-8ce0-b9a16f7d2e48'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:27.182843 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-b57e6056-90a3-4ac7-a9d8-7c8ced6f4852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7a00c9f55f74697af2b6f465a52dfcc'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:27.183869 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-b57e6056-90a3-4ac7-a9d8-7c8ced6f4852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7a00c9f55f74697af2b6f465a52dfcc'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:27.184930 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-b57e6056-90a3-4ac7-a9d8-7c8ced6f4852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7a00c9f55f74697af2b6f465a52dfcc'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:27.206250 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-b57e6056-90a3-4ac7-a9d8-7c8ced6f4852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7a00c9f55f74697af2b6f465a52dfcc'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:27.300566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-984854d3-93e6-4540-8ce0-b9a16f7d2e48 admin admin] The token cache is being invalidate because project 389e3c05babf477fad849535063df2ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:26:27.300912 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-984854d3-93e6-4540-8ce0-b9a16f7d2e48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:27.301568 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-984854d3-93e6-4540-8ce0-b9a16f7d2e48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:27.303823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1121/2245] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:26:26 2022] DELETE /identity/v3/projects/389e3c05babf477fad849535063df2ce => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:27.307593 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984854d3-93e6-4540-8ce0-b9a16f7d2e48 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:27.309125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-984854d3-93e6-4540-8ce0-b9a16f7d2e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:27.382872 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1125/2246] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:26:26 2022] DELETE /identity/v3/users/f7a00c9f55f74697af2b6f465a52dfcc => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:26:27.386706 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984854d3-93e6-4540-8ce0-b9a16f7d2e48 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:27.387986 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a10d2-1d99-4254-9fe7-2cc43042b435 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:27.388280 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a10d2-1d99-4254-9fe7-2cc43042b435 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:27.389095 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a10d2-1d99-4254-9fe7-2cc43042b435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:27.419661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-818a10d2-1d99-4254-9fe7-2cc43042b435 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:27.420936 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-818a10d2-1d99-4254-9fe7-2cc43042b435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:27.483376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b06d3-fabd-42ce-8bda-3b7305a01895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:27.485007 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b06d3-fabd-42ce-8bda-3b7305a01895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:27.485341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b06d3-fabd-42ce-8bda-3b7305a01895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:27.492367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b06d3-fabd-42ce-8bda-3b7305a01895 admin admin] PATH_INFO: `/v3/users/5abab218b32a43c9996f8791009415a7` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:27.492367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b06d3-fabd-42ce-8bda-3b7305a01895 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5abab218b32a43c9996f8791009415a7)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:27.493177 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b06d3-fabd-42ce-8bda-3b7305a01895 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:27.511354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1126/2247] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:26:27.514160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a10d2-1d99-4254-9fe7-2cc43042b435 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:27.521487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-818a10d2-1d99-4254-9fe7-2cc43042b435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:27.542969 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a10d2-1d99-4254-9fe7-2cc43042b435 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:27.545361 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:27.553045 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832e070-c6e7-4a7a-95e9-4acf971e6e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:27.554030 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832e070-c6e7-4a7a-95e9-4acf971e6e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:27.554417 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832e070-c6e7-4a7a-95e9-4acf971e6e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:27.554776 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832e070-c6e7-4a7a-95e9-4acf971e6e3a admin admin] PATH_INFO: `/v3/users/3c54fb85f05047b6a8b3ae0be7cc64f2` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:27.556508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a832e070-c6e7-4a7a-95e9-4acf971e6e3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c54fb85f05047b6a8b3ae0be7cc64f2)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:27.558642 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a832e070-c6e7-4a7a-95e9-4acf971e6e3a admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:27.689599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:27.823318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ff9b06d3-fabd-42ce-8bda-3b7305a01895 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5abab218b32a43c9996f8791009415a7'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:27.837013 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ff9b06d3-fabd-42ce-8bda-3b7305a01895 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5abab218b32a43c9996f8791009415a7'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:27.838357 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ff9b06d3-fabd-42ce-8bda-3b7305a01895 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5abab218b32a43c9996f8791009415a7'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:27.839169 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ff9b06d3-fabd-42ce-8bda-3b7305a01895 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5abab218b32a43c9996f8791009415a7'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:27.851701 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-a832e070-c6e7-4a7a-95e9-4acf971e6e3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c54fb85f05047b6a8b3ae0be7cc64f2'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:27.854512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ff9b06d3-fabd-42ce-8bda-3b7305a01895 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5abab218b32a43c9996f8791009415a7'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:27.859107 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-a832e070-c6e7-4a7a-95e9-4acf971e6e3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c54fb85f05047b6a8b3ae0be7cc64f2'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:27.859874 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-a832e070-c6e7-4a7a-95e9-4acf971e6e3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c54fb85f05047b6a8b3ae0be7cc64f2'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:27.860538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-a832e070-c6e7-4a7a-95e9-4acf971e6e3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c54fb85f05047b6a8b3ae0be7cc64f2'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:27.872331 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-a832e070-c6e7-4a7a-95e9-4acf971e6e3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c54fb85f05047b6a8b3ae0be7cc64f2'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:28.024445 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1122/2248] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:26:27 2022] DELETE /identity/v3/users/5abab218b32a43c9996f8791009415a7 => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:28.035424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b06d3-fabd-42ce-8bda-3b7305a01895 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:28.040600 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-ff9b06d3-fabd-42ce-8bda-3b7305a01895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:28.133278 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b06d3-fabd-42ce-8bda-3b7305a01895 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:28.137770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b80ad3-f07a-49e8-885b-a4be1514268a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:28.138526 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b80ad3-f07a-49e8-885b-a4be1514268a service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:28.139469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b80ad3-f07a-49e8-885b-a4be1514268a service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:28.139758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b80ad3-f07a-49e8-885b-a4be1514268a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:28.141630 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-82b80ad3-f07a-49e8-885b-a4be1514268a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:28.195949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1127/2249] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:26:27 2022] DELETE /identity/v3/users/3c54fb85f05047b6a8b3ae0be7cc64f2 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:28.207687 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003fd35-9cb0-4020-900d-6d3f18bbe48e None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:28.207980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003fd35-9cb0-4020-900d-6d3f18bbe48e None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:28.208264 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003fd35-9cb0-4020-900d-6d3f18bbe48e None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:28.212612 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1128/2250] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jun 23 09:26:28 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:26:28.217156 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1c956-2c3a-4951-8fb2-d3fa96ab34cd None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:28.217520 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1c956-2c3a-4951-8fb2-d3fa96ab34cd None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:28.218046 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1c956-2c3a-4951-8fb2-d3fa96ab34cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:28.245029 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b80ad3-f07a-49e8-885b-a4be1514268a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:28.248867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b80ad3-f07a-49e8-885b-a4be1514268a service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:28.253668 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-71e1c956-2c3a-4951-8fb2-d3fa96ab34cd None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:28.255215 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-71e1c956-2c3a-4951-8fb2-d3fa96ab34cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:28.262521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1123/2251] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:28.264441 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e744d3-0ff0-47a1-b442-bda450423676 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:28.264747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e744d3-0ff0-47a1-b442-bda450423676 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:28.265032 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e744d3-0ff0-47a1-b442-bda450423676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:28.307593 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-40e744d3-0ff0-47a1-b442-bda450423676 None None] MFA Rules not processed for user `ff025cb706d243778468ae4570374690`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:28.308869 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-40e744d3-0ff0-47a1-b442-bda450423676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:28.360911 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1129/2252] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:28.375674 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1c956-2c3a-4951-8fb2-d3fa96ab34cd None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:28.379927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-71e1c956-2c3a-4951-8fb2-d3fa96ab34cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:28.406334 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1124/2253] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jun 23 09:26:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:28.419220 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e744d3-0ff0-47a1-b442-bda450423676 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:28.421113 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-40e744d3-0ff0-47a1-b442-bda450423676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:28.465635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1c956-2c3a-4951-8fb2-d3fa96ab34cd None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:28.471325 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51da807-313e-44a8-8d6e-383f174aab05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:28.472354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51da807-313e-44a8-8d6e-383f174aab05 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:28.472907 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51da807-313e-44a8-8d6e-383f174aab05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:28.473429 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51da807-313e-44a8-8d6e-383f174aab05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:28.491370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51da807-313e-44a8-8d6e-383f174aab05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:28.493193 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51da807-313e-44a8-8d6e-383f174aab05 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:28.504250 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e744d3-0ff0-47a1-b442-bda450423676 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:28.508573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1130/2254] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:26:28.508824 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf4766-f733-4802-9d78-7feb1719abd7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36a6a3190dad4948a505bbe4c7095699', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:28.509785 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf4766-f733-4802-9d78-7feb1719abd7 service glance] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:28.510011 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf4766-f733-4802-9d78-7feb1719abd7 service glance] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:28.510309 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf4766-f733-4802-9d78-7feb1719abd7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:28.524401 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf4766-f733-4802-9d78-7feb1719abd7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:28.526037 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf4766-f733-4802-9d78-7feb1719abd7 service glance] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:28.552717 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1125/2255] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:28.789970 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51da807-313e-44a8-8d6e-383f174aab05 service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:28.791447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-f51da807-313e-44a8-8d6e-383f174aab05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:28.804607 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51da807-313e-44a8-8d6e-383f174aab05 service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:28.809367 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d78641-e009-434d-810e-c034a8cf4f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:28.810261 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d78641-e009-434d-810e-c034a8cf4f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:28.811210 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d78641-e009-434d-810e-c034a8cf4f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:28.811477 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d78641-e009-434d-810e-c034a8cf4f95 admin admin] PATH_INFO: `/v3/projects/b9322efcce0f4b828c34c9de5d0739d8` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:28.812805 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d78641-e009-434d-810e-c034a8cf4f95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9322efcce0f4b828c34c9de5d0739d8)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:28.814696 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d78641-e009-434d-810e-c034a8cf4f95 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:28.831747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ef5f6-c708-4e91-93ac-8271b2304b0d None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:28.832203 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ef5f6-c708-4e91-93ac-8271b2304b0d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:28.833153 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ef5f6-c708-4e91-93ac-8271b2304b0d None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:28.835744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1126/2256] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jun 23 09:26:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:26:28.842769 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3d960-6cac-4a9d-952d-baa6809b6410 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:28.843574 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3d960-6cac-4a9d-952d-baa6809b6410 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:28.843965 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3d960-6cac-4a9d-952d-baa6809b6410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:28.864859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-bcd3d960-6cac-4a9d-952d-baa6809b6410 None None] MFA Rules not processed for user `daed809f511a43ed8d3f88eba10466b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:28.866133 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-bcd3d960-6cac-4a9d-952d-baa6809b6410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:28.872959 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1127/2257] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jun 23 09:26:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:28.893047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3d960-6cac-4a9d-952d-baa6809b6410 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:28.893047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-bcd3d960-6cac-4a9d-952d-baa6809b6410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:28.914420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3d960-6cac-4a9d-952d-baa6809b6410 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:28.918685 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdab2de-9f8d-4cb5-b016-5825dc5e597f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:28.919389 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdab2de-9f8d-4cb5-b016-5825dc5e597f service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:28.919701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdab2de-9f8d-4cb5-b016-5825dc5e597f service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:28.920489 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdab2de-9f8d-4cb5-b016-5825dc5e597f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:28.928634 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdab2de-9f8d-4cb5-b016-5825dc5e597f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:28.930121 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdab2de-9f8d-4cb5-b016-5825dc5e597f service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:28.938996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1128/2258] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:28.958404 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-23d78641-e009-434d-810e-c034a8cf4f95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9322efcce0f4b828c34c9de5d0739d8', 'request_id': 'req-23d78641-e009-434d-810e-c034a8cf4f95'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:28.964611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-23d78641-e009-434d-810e-c034a8cf4f95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9322efcce0f4b828c34c9de5d0739d8', 'request_id': 'req-23d78641-e009-434d-810e-c034a8cf4f95'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:28.977085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdab2de-9f8d-4cb5-b016-5825dc5e597f service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:28.978545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-5fdab2de-9f8d-4cb5-b016-5825dc5e597f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:28.996701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdab2de-9f8d-4cb5-b016-5825dc5e597f service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:29.062525 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a94be3-f9dd-41e9-a763-4ff3b4c10dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:29.063454 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a94be3-f9dd-41e9-a763-4ff3b4c10dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:29.063746 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a94be3-f9dd-41e9-a763-4ff3b4c10dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:29.064038 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a94be3-f9dd-41e9-a763-4ff3b4c10dde admin admin] PATH_INFO: `/v3/projects/abec122c2f0e4a50bfc0efe0e74b308f` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:29.065223 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a94be3-f9dd-41e9-a763-4ff3b4c10dde admin admin] RBAC: Authorizing `identity:delete_project(project_id=abec122c2f0e4a50bfc0efe0e74b308f)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:29.067254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a94be3-f9dd-41e9-a763-4ff3b4c10dde admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:29.068158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-23d78641-e009-434d-810e-c034a8cf4f95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9322efcce0f4b828c34c9de5d0739d8', 'request_id': 'req-23d78641-e009-434d-810e-c034a8cf4f95'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:29.080133 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-23d78641-e009-434d-810e-c034a8cf4f95 admin admin] The token cache is being invalidate because project b9322efcce0f4b828c34c9de5d0739d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:26:29.082425 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-23d78641-e009-434d-810e-c034a8cf4f95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:29.082425 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-23d78641-e009-434d-810e-c034a8cf4f95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:29.082763 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1131/2259] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:26:28 2022] DELETE /identity/v3/projects/b9322efcce0f4b828c34c9de5d0739d8 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:29.084355 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21745ebb-bc17-466d-a8fe-5002cc3f49fd None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:29.084646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21745ebb-bc17-466d-a8fe-5002cc3f49fd None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:29.084921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21745ebb-bc17-466d-a8fe-5002cc3f49fd None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:29.086519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1132/2260] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jun 23 09:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:26:29.087657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab377e8-6540-4fb1-a6b2-a4b7b154aebe None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:29.088041 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab377e8-6540-4fb1-a6b2-a4b7b154aebe None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:29.088326 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab377e8-6540-4fb1-a6b2-a4b7b154aebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:29.116907 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-bab377e8-6540-4fb1-a6b2-a4b7b154aebe None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:29.118201 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-bab377e8-6540-4fb1-a6b2-a4b7b154aebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:29.126386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1133/2261] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:29.127657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cf5ba-b721-4dc1-91f4-5e3e53e5562a None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:29.127948 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cf5ba-b721-4dc1-91f4-5e3e53e5562a None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:29.128225 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cf5ba-b721-4dc1-91f4-5e3e53e5562a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:29.156014 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-4d6cf5ba-b721-4dc1-91f4-5e3e53e5562a None None] MFA Rules not processed for user `ff025cb706d243778468ae4570374690`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:29.157212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-4d6cf5ba-b721-4dc1-91f4-5e3e53e5562a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:29.231824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1134/2262] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jun 23 09:26:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:29.232936 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d465e0-5792-4ed2-a53d-6fb795aa0154 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:29.233274 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d465e0-5792-4ed2-a53d-6fb795aa0154 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:29.233562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d465e0-5792-4ed2-a53d-6fb795aa0154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:29.262488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-a8d465e0-5792-4ed2-a53d-6fb795aa0154 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:29.262488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a8d465e0-5792-4ed2-a53d-6fb795aa0154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:29.262739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-00a94be3-f9dd-41e9-a763-4ff3b4c10dde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abec122c2f0e4a50bfc0efe0e74b308f', 'request_id': 'req-00a94be3-f9dd-41e9-a763-4ff3b4c10dde'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:29.269485 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-00a94be3-f9dd-41e9-a763-4ff3b4c10dde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abec122c2f0e4a50bfc0efe0e74b308f', 'request_id': 'req-00a94be3-f9dd-41e9-a763-4ff3b4c10dde'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:29.277865 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-00a94be3-f9dd-41e9-a763-4ff3b4c10dde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abec122c2f0e4a50bfc0efe0e74b308f', 'request_id': 'req-00a94be3-f9dd-41e9-a763-4ff3b4c10dde'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:29.320242 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1135/2263] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:29.321235 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d465e0-5792-4ed2-a53d-6fb795aa0154 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:29.322552 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a8d465e0-5792-4ed2-a53d-6fb795aa0154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:29.336658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d465e0-5792-4ed2-a53d-6fb795aa0154 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:29.341856 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89ef79-e2bb-4caf-8b95-61a08cbee774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:29.342552 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89ef79-e2bb-4caf-8b95-61a08cbee774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:29.342552 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89ef79-e2bb-4caf-8b95-61a08cbee774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:29.342819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89ef79-e2bb-4caf-8b95-61a08cbee774 admin admin] PATH_INFO: `/v3/projects/033e7a9eee0149388e1c0815ba8f3ae1` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:29.344051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89ef79-e2bb-4caf-8b95-61a08cbee774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=033e7a9eee0149388e1c0815ba8f3ae1)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:29.346015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89ef79-e2bb-4caf-8b95-61a08cbee774 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:29.378326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-00a94be3-f9dd-41e9-a763-4ff3b4c10dde admin admin] The token cache is being invalidate because project abec122c2f0e4a50bfc0efe0e74b308f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:26:29.378628 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-00a94be3-f9dd-41e9-a763-4ff3b4c10dde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:29.381478 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-00a94be3-f9dd-41e9-a763-4ff3b4c10dde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:29.383660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1129/2264] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:26:28 2022] DELETE /identity/v3/projects/abec122c2f0e4a50bfc0efe0e74b308f => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:26:29.384534 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a94be3-f9dd-41e9-a763-4ff3b4c10dde admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:29.386208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-00a94be3-f9dd-41e9-a763-4ff3b4c10dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:29.444113 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a94be3-f9dd-41e9-a763-4ff3b4c10dde admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:29.448424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de17a05-51d5-4110-9881-6836beed60cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:29.449192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de17a05-51d5-4110-9881-6836beed60cd service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:29.449555 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de17a05-51d5-4110-9881-6836beed60cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:29.449887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de17a05-51d5-4110-9881-6836beed60cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:29.452583 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7de17a05-51d5-4110-9881-6836beed60cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:29.520493 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1c89ef79-e2bb-4caf-8b95-61a08cbee774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '033e7a9eee0149388e1c0815ba8f3ae1', 'request_id': 'req-1c89ef79-e2bb-4caf-8b95-61a08cbee774'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:29.524288 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de17a05-51d5-4110-9881-6836beed60cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:29.531669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de17a05-51d5-4110-9881-6836beed60cd service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:29.531805 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1c89ef79-e2bb-4caf-8b95-61a08cbee774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '033e7a9eee0149388e1c0815ba8f3ae1', 'request_id': 'req-1c89ef79-e2bb-4caf-8b95-61a08cbee774'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:29.537076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1130/2265] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:29.537892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de17a05-51d5-4110-9881-6836beed60cd service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:29.539257 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7de17a05-51d5-4110-9881-6836beed60cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:29.554657 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de17a05-51d5-4110-9881-6836beed60cd service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:29.617412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf0ad3-1c2a-4812-a5fa-034a89429220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:29.618767 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf0ad3-1c2a-4812-a5fa-034a89429220 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:29.618767 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf0ad3-1c2a-4812-a5fa-034a89429220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:29.618767 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf0ad3-1c2a-4812-a5fa-034a89429220 admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:29.619427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf0ad3-1c2a-4812-a5fa-034a89429220 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:26:29.619736 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf0ad3-1c2a-4812-a5fa-034a89429220 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:29.621328 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf0ad3-1c2a-4812-a5fa-034a89429220 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:29.625847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1c89ef79-e2bb-4caf-8b95-61a08cbee774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '033e7a9eee0149388e1c0815ba8f3ae1', 'request_id': 'req-1c89ef79-e2bb-4caf-8b95-61a08cbee774'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:29.642559 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1c89ef79-e2bb-4caf-8b95-61a08cbee774 admin admin] The token cache is being invalidate because project 033e7a9eee0149388e1c0815ba8f3ae1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:26:29.642853 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1c89ef79-e2bb-4caf-8b95-61a08cbee774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:29.646362 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1c89ef79-e2bb-4caf-8b95-61a08cbee774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:29.648592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1136/2266] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:26:29 2022] DELETE /identity/v3/projects/033e7a9eee0149388e1c0815ba8f3ae1 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:29.649746 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89ef79-e2bb-4caf-8b95-61a08cbee774 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:29.652634 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-1c89ef79-e2bb-4caf-8b95-61a08cbee774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:29.656122 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1131/2267] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:26:29.660570 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf0ad3-1c2a-4812-a5fa-034a89429220 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:29.678556 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-4ebf0ad3-1c2a-4812-a5fa-034a89429220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:29.694055 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89ef79-e2bb-4caf-8b95-61a08cbee774 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:29.711592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac0417-e5be-4663-a564-fd3e8ea1b72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:29.713011 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac0417-e5be-4663-a564-fd3e8ea1b72f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:29.713300 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac0417-e5be-4663-a564-fd3e8ea1b72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:29.713694 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac0417-e5be-4663-a564-fd3e8ea1b72f admin admin] PATH_INFO: `/v3/users/9f7b1abf962e402a9237faabeac86ea5` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:29.714288 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf0ad3-1c2a-4812-a5fa-034a89429220 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:29.715165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac0417-e5be-4663-a564-fd3e8ea1b72f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f7b1abf962e402a9237faabeac86ea5)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:29.719112 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac0417-e5be-4663-a564-fd3e8ea1b72f admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:29.730950 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4389a0-fba1-4b43-b190-2a452833a933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:29.731679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4389a0-fba1-4b43-b190-2a452833a933 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:29.733049 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4389a0-fba1-4b43-b190-2a452833a933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:29.733049 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4389a0-fba1-4b43-b190-2a452833a933 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:29.733519 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4389a0-fba1-4b43-b190-2a452833a933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:29.735850 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4389a0-fba1-4b43-b190-2a452833a933 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:29.915453 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1132/2268] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:26:29 2022] POST /identity/v3/projects => generated 379 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:26:29.919396 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4389a0-fba1-4b43-b190-2a452833a933 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:29.982979 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4389a0-fba1-4b43-b190-2a452833a933 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:29.988600 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e546d8-5441-44ab-b51d-bc3c89bb7800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:29.989294 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e546d8-5441-44ab-b51d-bc3c89bb7800 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:29.989705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e546d8-5441-44ab-b51d-bc3c89bb7800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:29.989988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e546d8-5441-44ab-b51d-bc3c89bb7800 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:29.990698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e546d8-5441-44ab-b51d-bc3c89bb7800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:29.992755 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e546d8-5441-44ab-b51d-bc3c89bb7800 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:30.038113 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-a2ac0417-e5be-4663-a564-fd3e8ea1b72f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f7b1abf962e402a9237faabeac86ea5'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:30.045160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-a2ac0417-e5be-4663-a564-fd3e8ea1b72f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f7b1abf962e402a9237faabeac86ea5'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:30.045909 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-a2ac0417-e5be-4663-a564-fd3e8ea1b72f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f7b1abf962e402a9237faabeac86ea5'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:30.046558 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-a2ac0417-e5be-4663-a564-fd3e8ea1b72f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f7b1abf962e402a9237faabeac86ea5'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:30.066302 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-a2ac0417-e5be-4663-a564-fd3e8ea1b72f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f7b1abf962e402a9237faabeac86ea5'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:30.103034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1133/2269] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:26:29 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:26:30.107074 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e546d8-5441-44ab-b51d-bc3c89bb7800 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:30.109545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-31e546d8-5441-44ab-b51d-bc3c89bb7800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:30.121827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e546d8-5441-44ab-b51d-bc3c89bb7800 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:30.127144 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906dc8e-5c1f-4ebf-8300-9a8e6b460fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:30.127888 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906dc8e-5c1f-4ebf-8300-9a8e6b460fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:30.128456 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906dc8e-5c1f-4ebf-8300-9a8e6b460fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:30.128605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906dc8e-5c1f-4ebf-8300-9a8e6b460fc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:30.129213 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906dc8e-5c1f-4ebf-8300-9a8e6b460fc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:30.130800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906dc8e-5c1f-4ebf-8300-9a8e6b460fc9 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:30.141273 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1134/2270] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:26:30.145125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906dc8e-5c1f-4ebf-8300-9a8e6b460fc9 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:30.159859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906dc8e-5c1f-4ebf-8300-9a8e6b460fc9 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:30.164705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51b824-ca2f-4163-8be0-a89c58580350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:30.165380 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51b824-ca2f-4163-8be0-a89c58580350 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:30.169134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51b824-ca2f-4163-8be0-a89c58580350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:30.169134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51b824-ca2f-4163-8be0-a89c58580350 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:30.169134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51b824-ca2f-4163-8be0-a89c58580350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:30.169134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51b824-ca2f-4163-8be0-a89c58580350 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:30.182195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1135/2271] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:26:30.186747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51b824-ca2f-4163-8be0-a89c58580350 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:30.196834 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51b824-ca2f-4163-8be0-a89c58580350 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:30.199394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1137/2272] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:26:29 2022] DELETE /identity/v3/users/9f7b1abf962e402a9237faabeac86ea5 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:30.206936 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cbb73-0da7-473c-adc7-bb0b644e8ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:30.206936 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cbb73-0da7-473c-adc7-bb0b644e8ee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:30.206936 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cbb73-0da7-473c-adc7-bb0b644e8ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:30.206936 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cbb73-0da7-473c-adc7-bb0b644e8ee9 admin admin] PATH_INFO: `/v3/projects/14dfbc5cd87146368f0748ee84879b5b/users/6f9ad61c5eb5401c8c756a9766801517/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:30.218887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889cbb73-0da7-473c-adc7-bb0b644e8ee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14dfbc5cd87146368f0748ee84879b5b, user_id=6f9ad61c5eb5401c8c756a9766801517, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:30.221709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889cbb73-0da7-473c-adc7-bb0b644e8ee9 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:30.330516 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1136/2273] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:26:30 2022] PUT /identity/v3/projects/14dfbc5cd87146368f0748ee84879b5b/users/6f9ad61c5eb5401c8c756a9766801517/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:26:30.340730 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625685a-225b-42fa-bbdf-da5aa10ea8ac None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:30.341030 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625685a-225b-42fa-bbdf-da5aa10ea8ac None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:30.341310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625685a-225b-42fa-bbdf-da5aa10ea8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:30.374498 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-8625685a-225b-42fa-bbdf-da5aa10ea8ac None None] MFA Rules not processed for user `6f9ad61c5eb5401c8c756a9766801517`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:30.375857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-8625685a-225b-42fa-bbdf-da5aa10ea8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:30.446186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1138/2274] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:30.450646 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cbb73-0da7-473c-adc7-bb0b644e8ee9 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:30.527300 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cbb73-0da7-473c-adc7-bb0b644e8ee9 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:30.532818 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd20d5c-d495-430e-9a91-7f3e766e1ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:30.533938 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd20d5c-d495-430e-9a91-7f3e766e1ba5 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:30.534338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd20d5c-d495-430e-9a91-7f3e766e1ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:30.534712 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd20d5c-d495-430e-9a91-7f3e766e1ba5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:30.535706 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd20d5c-d495-430e-9a91-7f3e766e1ba5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:30.537639 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd20d5c-d495-430e-9a91-7f3e766e1ba5 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:30.552101 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:30.718513 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1137/2275] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:26:30 2022] POST /identity/v3/projects => generated 377 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:26:30.723005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625685a-225b-42fa-bbdf-da5aa10ea8ac None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:30.726076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:30.736305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625685a-225b-42fa-bbdf-da5aa10ea8ac None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:30.798770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f41a6-267f-4c27-b563-720cbe6def55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:30.805712 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f41a6-267f-4c27-b563-720cbe6def55 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:30.805712 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f41a6-267f-4c27-b563-720cbe6def55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:30.805712 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f41a6-267f-4c27-b563-720cbe6def55 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:30.805712 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f41a6-267f-4c27-b563-720cbe6def55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:30.805712 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f41a6-267f-4c27-b563-720cbe6def55 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:30.886781 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd20d5c-d495-430e-9a91-7f3e766e1ba5 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:30.890452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-1dd20d5c-d495-430e-9a91-7f3e766e1ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:30.901033 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd20d5c-d495-430e-9a91-7f3e766e1ba5 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:30.906165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d0594-67a1-4f93-ab5a-02666eee3fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:30.907151 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d0594-67a1-4f93-ab5a-02666eee3fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:30.907442 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d0594-67a1-4f93-ab5a-02666eee3fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:30.907720 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d0594-67a1-4f93-ab5a-02666eee3fab admin admin] PATH_INFO: `/v3/projects/bc47b6fc71bf4c2192459dea71872b16` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:30.909016 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d0594-67a1-4f93-ab5a-02666eee3fab admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc47b6fc71bf4c2192459dea71872b16)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:30.910979 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d0594-67a1-4f93-ab5a-02666eee3fab admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:30.921972 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1139/2276] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:26:30 2022] POST /identity/v3/users => generated 289 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:26:30.925762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f41a6-267f-4c27-b563-720cbe6def55 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:30.935772 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f41a6-267f-4c27-b563-720cbe6def55 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:30.940404 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105c833-f648-42f1-8370-7cbbf4f29825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:30.941054 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105c833-f648-42f1-8370-7cbbf4f29825 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:30.941338 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105c833-f648-42f1-8370-7cbbf4f29825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:30.941611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105c833-f648-42f1-8370-7cbbf4f29825 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:30.942332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105c833-f648-42f1-8370-7cbbf4f29825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:30.943950 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105c833-f648-42f1-8370-7cbbf4f29825 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:30.953781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1140/2277] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:26:30.957586 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105c833-f648-42f1-8370-7cbbf4f29825 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:30.970303 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105c833-f648-42f1-8370-7cbbf4f29825 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:30.976382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3572c-0266-4728-b665-4edf44556e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:30.977053 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3572c-0266-4728-b665-4edf44556e71 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:30.978001 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3572c-0266-4728-b665-4edf44556e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:30.978290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3572c-0266-4728-b665-4edf44556e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:30.979044 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3572c-0266-4728-b665-4edf44556e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:30.980667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3572c-0266-4728-b665-4edf44556e71 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:30.992088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1141/2278] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:26:30.995842 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3572c-0266-4728-b665-4edf44556e71 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:31.005151 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3572c-0266-4728-b665-4edf44556e71 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:31.009645 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a8c16-a2e8-4e9b-b5dc-f3670026cf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:31.010474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a8c16-a2e8-4e9b-b5dc-f3670026cf46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:31.010757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a8c16-a2e8-4e9b-b5dc-f3670026cf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:31.011075 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a8c16-a2e8-4e9b-b5dc-f3670026cf46 admin admin] PATH_INFO: `/v3/projects/ba19986bddee45f0a79de1d995a07872/users/768cb73d7b21456d97b14b1f968513de/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:31.025215 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a8c16-a2e8-4e9b-b5dc-f3670026cf46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba19986bddee45f0a79de1d995a07872, user_id=768cb73d7b21456d97b14b1f968513de, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:31.027817 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a8c16-a2e8-4e9b-b5dc-f3670026cf46 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:31.038632 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e87d0594-67a1-4f93-ab5a-02666eee3fab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc47b6fc71bf4c2192459dea71872b16', 'request_id': 'req-e87d0594-67a1-4f93-ab5a-02666eee3fab'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:31.046014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e87d0594-67a1-4f93-ab5a-02666eee3fab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc47b6fc71bf4c2192459dea71872b16', 'request_id': 'req-e87d0594-67a1-4f93-ab5a-02666eee3fab'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:31.051723 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e87d0594-67a1-4f93-ab5a-02666eee3fab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc47b6fc71bf4c2192459dea71872b16', 'request_id': 'req-e87d0594-67a1-4f93-ab5a-02666eee3fab'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:31.130199 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e87d0594-67a1-4f93-ab5a-02666eee3fab admin admin] The token cache is being invalidate because project bc47b6fc71bf4c2192459dea71872b16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:26:31.130493 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e87d0594-67a1-4f93-ab5a-02666eee3fab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:31.131244 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e87d0594-67a1-4f93-ab5a-02666eee3fab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:31.131750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1142/2279] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:26:30 2022] PUT /identity/v3/projects/ba19986bddee45f0a79de1d995a07872/users/768cb73d7b21456d97b14b1f968513de/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:26:31.132869 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1138/2280] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:26:30 2022] DELETE /identity/v3/projects/bc47b6fc71bf4c2192459dea71872b16 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:26:31.136753 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a8c16-a2e8-4e9b-b5dc-f3670026cf46 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:31.138991 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-9e1a8c16-a2e8-4e9b-b5dc-f3670026cf46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:31.148509 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe2a96-a7f6-431d-acaf-21dab32b3b87 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:31.148831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe2a96-a7f6-431d-acaf-21dab32b3b87 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:31.149089 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe2a96-a7f6-431d-acaf-21dab32b3b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:31.178782 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-7abe2a96-a7f6-431d-acaf-21dab32b3b87 None None] MFA Rules not processed for user `768cb73d7b21456d97b14b1f968513de`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:31.180042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7abe2a96-a7f6-431d-acaf-21dab32b3b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:31.221985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a8c16-a2e8-4e9b-b5dc-f3670026cf46 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:31.226800 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5649d15-ac2b-41ab-bf90-03f41d6155be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:31.227706 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5649d15-ac2b-41ab-bf90-03f41d6155be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:31.227991 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5649d15-ac2b-41ab-bf90-03f41d6155be admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:31.228268 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5649d15-ac2b-41ab-bf90-03f41d6155be admin admin] PATH_INFO: `/v3/users/a2c1e681c40f406085793ea5d75163c1` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:31.229527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5649d15-ac2b-41ab-bf90-03f41d6155be admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2c1e681c40f406085793ea5d75163c1)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:31.231509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5649d15-ac2b-41ab-bf90-03f41d6155be admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:31.245202 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1139/2281] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:31.249650 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe2a96-a7f6-431d-acaf-21dab32b3b87 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:31.250998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7abe2a96-a7f6-431d-acaf-21dab32b3b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:31.264525 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe2a96-a7f6-431d-acaf-21dab32b3b87 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:31.269141 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dee5aa-6fe0-49ad-8d4f-1d9d76643e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:31.269794 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dee5aa-6fe0-49ad-8d4f-1d9d76643e55 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:31.270085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dee5aa-6fe0-49ad-8d4f-1d9d76643e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:31.270367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dee5aa-6fe0-49ad-8d4f-1d9d76643e55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:31.271088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dee5aa-6fe0-49ad-8d4f-1d9d76643e55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:31.272984 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dee5aa-6fe0-49ad-8d4f-1d9d76643e55 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:31.390892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1140/2282] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:26:31 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:26:31.394395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dee5aa-6fe0-49ad-8d4f-1d9d76643e55 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:31.454675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-a5649d15-ac2b-41ab-bf90-03f41d6155be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2c1e681c40f406085793ea5d75163c1'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:31.456693 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dee5aa-6fe0-49ad-8d4f-1d9d76643e55 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:31.461449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-a5649d15-ac2b-41ab-bf90-03f41d6155be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2c1e681c40f406085793ea5d75163c1'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:31.461624 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8a611-e6f1-4d28-906d-7c0ab934d6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:31.462119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-a5649d15-ac2b-41ab-bf90-03f41d6155be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2c1e681c40f406085793ea5d75163c1'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:31.462414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8a611-e6f1-4d28-906d-7c0ab934d6b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:31.462706 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8a611-e6f1-4d28-906d-7c0ab934d6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:31.462818 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-a5649d15-ac2b-41ab-bf90-03f41d6155be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2c1e681c40f406085793ea5d75163c1'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:31.463016 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8a611-e6f1-4d28-906d-7c0ab934d6b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:31.463721 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8a611-e6f1-4d28-906d-7c0ab934d6b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:31.465705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8a611-e6f1-4d28-906d-7c0ab934d6b9 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:31.475085 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-a5649d15-ac2b-41ab-bf90-03f41d6155be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2c1e681c40f406085793ea5d75163c1'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:31.604993 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1143/2283] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:26:31 2022] DELETE /identity/v3/users/a2c1e681c40f406085793ea5d75163c1 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:31.608307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1141/2284] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:26:31 2022] POST /identity/v3/users => generated 295 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:26:31.609583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5649d15-ac2b-41ab-bf90-03f41d6155be admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:31.611040 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a5649d15-ac2b-41ab-bf90-03f41d6155be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:31.681048 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5649d15-ac2b-41ab-bf90-03f41d6155be admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:31.687212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e74a9-d444-407a-86f6-dcc0df54b658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:31.687701 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e74a9-d444-407a-86f6-dcc0df54b658 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:31.687988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e74a9-d444-407a-86f6-dcc0df54b658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:31.688267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e74a9-d444-407a-86f6-dcc0df54b658 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:31.689009 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e74a9-d444-407a-86f6-dcc0df54b658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:31.690748 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e74a9-d444-407a-86f6-dcc0df54b658 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:31.701758 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1144/2285] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:26:31.706304 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8a611-e6f1-4d28-906d-7c0ab934d6b9 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:31.718780 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8a611-e6f1-4d28-906d-7c0ab934d6b9 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:31.723986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe34f13-3db5-49e8-a15d-6337a00242b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:31.725035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe34f13-3db5-49e8-a15d-6337a00242b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:31.725413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe34f13-3db5-49e8-a15d-6337a00242b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:31.725800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe34f13-3db5-49e8-a15d-6337a00242b8 admin admin] PATH_INFO: `/v3/projects/1753f2c03fee482fb55bb4d41654393f/users/d490bac4439d45bfa8453b1e238a36fe/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:31.746196 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe34f13-3db5-49e8-a15d-6337a00242b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1753f2c03fee482fb55bb4d41654393f, user_id=d490bac4439d45bfa8453b1e238a36fe, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:31.746196 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe34f13-3db5-49e8-a15d-6337a00242b8 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:32.006712 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1142/2286] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:26:31 2022] PUT /identity/v3/projects/1753f2c03fee482fb55bb4d41654393f/users/d490bac4439d45bfa8453b1e238a36fe/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:32.010906 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e74a9-d444-407a-86f6-dcc0df54b658 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:32.021343 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e74a9-d444-407a-86f6-dcc0df54b658 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:32.099001 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef01fb2-7ddb-429c-8bd9-069a6c8470af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:32.099001 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef01fb2-7ddb-429c-8bd9-069a6c8470af admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:32.099001 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef01fb2-7ddb-429c-8bd9-069a6c8470af admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:32.099001 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef01fb2-7ddb-429c-8bd9-069a6c8470af admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:32.099413 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef01fb2-7ddb-429c-8bd9-069a6c8470af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:32.100977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef01fb2-7ddb-429c-8bd9-069a6c8470af admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:32.113443 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1145/2287] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:26:32.118123 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe34f13-3db5-49e8-a15d-6337a00242b8 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:32.140224 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe34f13-3db5-49e8-a15d-6337a00242b8 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:32.145752 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecb50a-e355-49f2-afb6-f549f97dad7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:32.146748 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecb50a-e355-49f2-afb6-f549f97dad7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:32.147069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecb50a-e355-49f2-afb6-f549f97dad7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:32.147349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecb50a-e355-49f2-afb6-f549f97dad7a admin admin] PATH_INFO: `/v3/projects/1753f2c03fee482fb55bb4d41654393f/users/d490bac4439d45bfa8453b1e238a36fe/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:32.153235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ecb50a-e355-49f2-afb6-f549f97dad7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1753f2c03fee482fb55bb4d41654393f, user_id=d490bac4439d45bfa8453b1e238a36fe, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:32.156072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ecb50a-e355-49f2-afb6-f549f97dad7a admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:32.168230 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1143/2288] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:26:32 2022] PUT /identity/v3/projects/1753f2c03fee482fb55bb4d41654393f/users/d490bac4439d45bfa8453b1e238a36fe/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:32.179647 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1baaf25-02b8-4c72-b376-c7f3cdd2f78d None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:32.180224 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1baaf25-02b8-4c72-b376-c7f3cdd2f78d None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:32.180324 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1baaf25-02b8-4c72-b376-c7f3cdd2f78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:32.209643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-d1baaf25-02b8-4c72-b376-c7f3cdd2f78d None None] MFA Rules not processed for user `d490bac4439d45bfa8453b1e238a36fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:32.210877 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d1baaf25-02b8-4c72-b376-c7f3cdd2f78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:32.299644 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1146/2289] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:26:32.313015 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecb50a-e355-49f2-afb6-f549f97dad7a admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:32.315026 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-49ecb50a-e355-49f2-afb6-f549f97dad7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:32.374873 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecb50a-e355-49f2-afb6-f549f97dad7a admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:32.378617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038429ab-3ec7-4ab4-96b1-489393a7d4b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:32.379274 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038429ab-3ec7-4ab4-96b1-489393a7d4b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:32.379559 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038429ab-3ec7-4ab4-96b1-489393a7d4b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:32.379836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038429ab-3ec7-4ab4-96b1-489393a7d4b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:32.389174 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038429ab-3ec7-4ab4-96b1-489393a7d4b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:32.390611 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038429ab-3ec7-4ab4-96b1-489393a7d4b8 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:32.400524 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1144/2290] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:26:32.533786 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1baaf25-02b8-4c72-b376-c7f3cdd2f78d None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:32.535210 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d1baaf25-02b8-4c72-b376-c7f3cdd2f78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:32.546973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1baaf25-02b8-4c72-b376-c7f3cdd2f78d None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:32.552749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a7b22-5198-4552-a1b3-649b822b56b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:32.552749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a7b22-5198-4552-a1b3-649b822b56b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:32.552749 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a7b22-5198-4552-a1b3-649b822b56b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:32.553303 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a7b22-5198-4552-a1b3-649b822b56b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:32.555701 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-dd6a7b22-5198-4552-a1b3-649b822b56b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:32.566601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038429ab-3ec7-4ab4-96b1-489393a7d4b8 service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:32.570555 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-038429ab-3ec7-4ab4-96b1-489393a7d4b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:32.646504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a7b22-5198-4552-a1b3-649b822b56b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:32.646504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a7b22-5198-4552-a1b3-649b822b56b6 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:32.651463 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038429ab-3ec7-4ab4-96b1-489393a7d4b8 service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:32.653750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1147/2291] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:32.656651 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9fa82-5f5f-4312-9a24-fc9fc02fbfc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:32.657518 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9fa82-5f5f-4312-9a24-fc9fc02fbfc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:32.657813 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9fa82-5f5f-4312-9a24-fc9fc02fbfc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:32.658092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9fa82-5f5f-4312-9a24-fc9fc02fbfc6 admin admin] PATH_INFO: `/v3/projects/000776dc041d460e9f482155fb819094` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:32.659424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9fa82-5f5f-4312-9a24-fc9fc02fbfc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=000776dc041d460e9f482155fb819094)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:32.661273 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9fa82-5f5f-4312-9a24-fc9fc02fbfc6 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:32.808028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-31e9fa82-5f5f-4312-9a24-fc9fc02fbfc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '000776dc041d460e9f482155fb819094', 'request_id': 'req-31e9fa82-5f5f-4312-9a24-fc9fc02fbfc6'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:32.814120 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-31e9fa82-5f5f-4312-9a24-fc9fc02fbfc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '000776dc041d460e9f482155fb819094', 'request_id': 'req-31e9fa82-5f5f-4312-9a24-fc9fc02fbfc6'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:32.819597 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-31e9fa82-5f5f-4312-9a24-fc9fc02fbfc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '000776dc041d460e9f482155fb819094', 'request_id': 'req-31e9fa82-5f5f-4312-9a24-fc9fc02fbfc6'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:32.930713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-31e9fa82-5f5f-4312-9a24-fc9fc02fbfc6 admin admin] The token cache is being invalidate because project 000776dc041d460e9f482155fb819094 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:26:32.931099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-31e9fa82-5f5f-4312-9a24-fc9fc02fbfc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:32.933435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-31e9fa82-5f5f-4312-9a24-fc9fc02fbfc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:32.933435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1145/2292] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:26:32 2022] DELETE /identity/v3/projects/000776dc041d460e9f482155fb819094 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:26:32.940355 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e41a0-319f-40b3-81a5-55457c46eb14 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:32.941127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e41a0-319f-40b3-81a5-55457c46eb14 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:32.941479 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e41a0-319f-40b3-81a5-55457c46eb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:32.969599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-531e41a0-319f-40b3-81a5-55457c46eb14 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:32.971050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-531e41a0-319f-40b3-81a5-55457c46eb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:33.043336 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1148/2293] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:26:33.052019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7383d1-15eb-466d-92c9-8f7acfec9881 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:33.052311 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7383d1-15eb-466d-92c9-8f7acfec9881 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:33.052796 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7383d1-15eb-466d-92c9-8f7acfec9881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:33.078598 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-ce7383d1-15eb-466d-92c9-8f7acfec9881 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:33.078598 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-ce7383d1-15eb-466d-92c9-8f7acfec9881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:33.087018 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1146/2294] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:33.091920 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e41a0-319f-40b3-81a5-55457c46eb14 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:33.103016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e41a0-319f-40b3-81a5-55457c46eb14 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:33.113185 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564570e-bfcb-484d-8bfc-884373b73a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:33.114535 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564570e-bfcb-484d-8bfc-884373b73a7b admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:33.115027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564570e-bfcb-484d-8bfc-884373b73a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:33.115460 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564570e-bfcb-484d-8bfc-884373b73a7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:33.116697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564570e-bfcb-484d-8bfc-884373b73a7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:26:33.117745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564570e-bfcb-484d-8bfc-884373b73a7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:33.120711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564570e-bfcb-484d-8bfc-884373b73a7b admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:33.153935 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1149/2295] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:26:33.155670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7383d1-15eb-466d-92c9-8f7acfec9881 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:33.173619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7383d1-15eb-466d-92c9-8f7acfec9881 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:33.179117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812d168-eced-4c32-83c7-40e46b1aeb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:33.179811 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812d168-eced-4c32-83c7-40e46b1aeb0e admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:33.180228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812d168-eced-4c32-83c7-40e46b1aeb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:33.180520 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812d168-eced-4c32-83c7-40e46b1aeb0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:33.181368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812d168-eced-4c32-83c7-40e46b1aeb0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:33.183322 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812d168-eced-4c32-83c7-40e46b1aeb0e admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:33.314218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1147/2296] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:26:33 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:26:33.320213 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564570e-bfcb-484d-8bfc-884373b73a7b admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:33.332524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564570e-bfcb-484d-8bfc-884373b73a7b admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:33.398607 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58a9c2-68d2-42ec-b1cc-6662814c2dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:33.399333 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58a9c2-68d2-42ec-b1cc-6662814c2dbb admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:33.399621 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58a9c2-68d2-42ec-b1cc-6662814c2dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:33.399930 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58a9c2-68d2-42ec-b1cc-6662814c2dbb admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:33.400635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58a9c2-68d2-42ec-b1cc-6662814c2dbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:33.402623 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58a9c2-68d2-42ec-b1cc-6662814c2dbb admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:33.538340 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1150/2297] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:26:33 2022] POST /identity/v3/users => generated 287 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:26:33.542513 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812d168-eced-4c32-83c7-40e46b1aeb0e admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:33.553247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812d168-eced-4c32-83c7-40e46b1aeb0e admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:33.557840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4777dbd-d4a3-4917-b82a-2437d234ca63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:33.558510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4777dbd-d4a3-4917-b82a-2437d234ca63 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:33.558805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4777dbd-d4a3-4917-b82a-2437d234ca63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:33.562252 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4777dbd-d4a3-4917-b82a-2437d234ca63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:33.562252 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4777dbd-d4a3-4917-b82a-2437d234ca63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:33.562252 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4777dbd-d4a3-4917-b82a-2437d234ca63 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:33.572825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1148/2298] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:26:33.576885 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58a9c2-68d2-42ec-b1cc-6662814c2dbb admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:33.591374 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58a9c2-68d2-42ec-b1cc-6662814c2dbb admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:33.598734 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8bb06-20be-42cc-9042-f402973ca637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:33.599442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8bb06-20be-42cc-9042-f402973ca637 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:33.599735 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8bb06-20be-42cc-9042-f402973ca637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:33.600017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8bb06-20be-42cc-9042-f402973ca637 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:33.600755 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8bb06-20be-42cc-9042-f402973ca637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:33.602501 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8bb06-20be-42cc-9042-f402973ca637 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:33.614185 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1151/2299] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:26:33.618298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4777dbd-d4a3-4917-b82a-2437d234ca63 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:33.629173 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4777dbd-d4a3-4917-b82a-2437d234ca63 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:33.633850 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33596829-0946-486e-bf57-bf6798395213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:33.636939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33596829-0946-486e-bf57-bf6798395213 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:33.636939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33596829-0946-486e-bf57-bf6798395213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:33.636939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33596829-0946-486e-bf57-bf6798395213 admin admin] PATH_INFO: `/v3/projects/3ee3f1c33377446489b75e9faaa5654b/users/65220457c29c46358fee054fb9366406/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:33.662304 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33596829-0946-486e-bf57-bf6798395213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee3f1c33377446489b75e9faaa5654b, user_id=65220457c29c46358fee054fb9366406, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:33.665006 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33596829-0946-486e-bf57-bf6798395213 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:33.874629 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1149/2300] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:26:33 2022] PUT /identity/v3/projects/3ee3f1c33377446489b75e9faaa5654b/users/65220457c29c46358fee054fb9366406/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:33.876360 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7e2c3-a0ca-4b70-8a88-16af8c7ec64c None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:33.876653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7e2c3-a0ca-4b70-8a88-16af8c7ec64c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:33.877978 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7e2c3-a0ca-4b70-8a88-16af8c7ec64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:33.896424 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-a4d7e2c3-a0ca-4b70-8a88-16af8c7ec64c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:33.900244 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a4d7e2c3-a0ca-4b70-8a88-16af8c7ec64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:33.985836 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1152/2301] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:33.997648 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33596829-0946-486e-bf57-bf6798395213 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:33.999616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-33596829-0946-486e-bf57-bf6798395213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:34.097239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33596829-0946-486e-bf57-bf6798395213 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:34.101074 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5113db7-71a3-49fe-a6d9-fff8c10f95c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:34.101746 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5113db7-71a3-49fe-a6d9-fff8c10f95c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:34.102044 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5113db7-71a3-49fe-a6d9-fff8c10f95c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:34.102331 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5113db7-71a3-49fe-a6d9-fff8c10f95c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:34.113646 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5113db7-71a3-49fe-a6d9-fff8c10f95c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:34.115422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5113db7-71a3-49fe-a6d9-fff8c10f95c9 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:34.126875 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1150/2302] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:34.880321 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:35.189369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:35.706875 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88242be0-046b-47fa-9296-c94d1d03a00b None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:35.707209 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88242be0-046b-47fa-9296-c94d1d03a00b None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:35.707492 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88242be0-046b-47fa-9296-c94d1d03a00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:35.727329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-88242be0-046b-47fa-9296-c94d1d03a00b None None] MFA Rules not processed for user `cfbaba84402d4c31bafcd12133d74c03`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:35.729195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-88242be0-046b-47fa-9296-c94d1d03a00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:35.836791 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1153/2303] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jun 23 09:26:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:35.850057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5113db7-71a3-49fe-a6d9-fff8c10f95c9 service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:35.852739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-c5113db7-71a3-49fe-a6d9-fff8c10f95c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:35.941054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5113db7-71a3-49fe-a6d9-fff8c10f95c9 service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:35.946321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2476b-f15e-4b5c-b411-a518ed2042cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b1331b4f0924bb8b137a22a33c9d130', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:35.946955 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2476b-f15e-4b5c-b411-a518ed2042cb service swift] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:35.947252 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2476b-f15e-4b5c-b411-a518ed2042cb service swift] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:35.947554 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2476b-f15e-4b5c-b411-a518ed2042cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:35.956304 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c2476b-f15e-4b5c-b411-a518ed2042cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:35.957926 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c2476b-f15e-4b5c-b411-a518ed2042cb service swift] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:35.968640 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1151/2304] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jun 23 09:26:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 09:26:37.377095 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88242be0-046b-47fa-9296-c94d1d03a00b None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:37.378750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-88242be0-046b-47fa-9296-c94d1d03a00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:37.418662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88242be0-046b-47fa-9296-c94d1d03a00b None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:37.437432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b058ab-288a-47ab-8c81-4951fc4606cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:37.438314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b058ab-288a-47ab-8c81-4951fc4606cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:37.438601 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b058ab-288a-47ab-8c81-4951fc4606cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:37.438878 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b058ab-288a-47ab-8c81-4951fc4606cb admin admin] PATH_INFO: `/v3/users/6f9ad61c5eb5401c8c756a9766801517` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:37.440722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b058ab-288a-47ab-8c81-4951fc4606cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f9ad61c5eb5401c8c756a9766801517)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:37.442737 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b058ab-288a-47ab-8c81-4951fc4606cb admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:37.796625 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-71b058ab-288a-47ab-8c81-4951fc4606cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f9ad61c5eb5401c8c756a9766801517'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:37.814822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-71b058ab-288a-47ab-8c81-4951fc4606cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f9ad61c5eb5401c8c756a9766801517'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:37.816496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-71b058ab-288a-47ab-8c81-4951fc4606cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f9ad61c5eb5401c8c756a9766801517'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:37.823323 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-71b058ab-288a-47ab-8c81-4951fc4606cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f9ad61c5eb5401c8c756a9766801517'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:37.846049 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-71b058ab-288a-47ab-8c81-4951fc4606cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f9ad61c5eb5401c8c756a9766801517'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:38.006213 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1154/2305] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:26:37 2022] DELETE /identity/v3/users/6f9ad61c5eb5401c8c756a9766801517 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:38.010629 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29f4fe-ddb4-4eb6-b36e-c5be4ea226ca None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:38.011195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29f4fe-ddb4-4eb6-b36e-c5be4ea226ca None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:38.011421 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29f4fe-ddb4-4eb6-b36e-c5be4ea226ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:38.050370 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-4b29f4fe-ddb4-4eb6-b36e-c5be4ea226ca None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:38.052891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-4b29f4fe-ddb4-4eb6-b36e-c5be4ea226ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:38.177583 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1152/2306] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:38.187713 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b058ab-288a-47ab-8c81-4951fc4606cb admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:38.190745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-71b058ab-288a-47ab-8c81-4951fc4606cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:38.303411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b058ab-288a-47ab-8c81-4951fc4606cb admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:38.305954 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c170a-3e37-468b-b77e-738264a2e69d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:38.306730 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c170a-3e37-468b-b77e-738264a2e69d service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:38.307103 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c170a-3e37-468b-b77e-738264a2e69d service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:38.307599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c170a-3e37-468b-b77e-738264a2e69d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:38.336255 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c170a-3e37-468b-b77e-738264a2e69d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:38.337888 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c170a-3e37-468b-b77e-738264a2e69d service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:38.353922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1155/2307] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:38.433919 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:38.545447 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:39.096042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29f4fe-ddb4-4eb6-b36e-c5be4ea226ca None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:39.097427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-4b29f4fe-ddb4-4eb6-b36e-c5be4ea226ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:39.126755 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29f4fe-ddb4-4eb6-b36e-c5be4ea226ca None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:39.132924 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a874d-9a79-4570-bad1-f1ff02e85dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:39.134302 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a874d-9a79-4570-bad1-f1ff02e85dba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:39.135969 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a874d-9a79-4570-bad1-f1ff02e85dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:39.135969 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a874d-9a79-4570-bad1-f1ff02e85dba admin admin] PATH_INFO: `/v3/projects/14dfbc5cd87146368f0748ee84879b5b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:39.137289 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a874d-9a79-4570-bad1-f1ff02e85dba admin admin] RBAC: Authorizing `identity:delete_project(project_id=14dfbc5cd87146368f0748ee84879b5b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:39.150960 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a874d-9a79-4570-bad1-f1ff02e85dba admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:40.444000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e81a874d-9a79-4570-bad1-f1ff02e85dba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14dfbc5cd87146368f0748ee84879b5b', 'request_id': 'req-e81a874d-9a79-4570-bad1-f1ff02e85dba'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:40.457038 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e81a874d-9a79-4570-bad1-f1ff02e85dba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14dfbc5cd87146368f0748ee84879b5b', 'request_id': 'req-e81a874d-9a79-4570-bad1-f1ff02e85dba'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:40.466210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e81a874d-9a79-4570-bad1-f1ff02e85dba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14dfbc5cd87146368f0748ee84879b5b', 'request_id': 'req-e81a874d-9a79-4570-bad1-f1ff02e85dba'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:40.682358 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e81a874d-9a79-4570-bad1-f1ff02e85dba admin admin] The token cache is being invalidate because project 14dfbc5cd87146368f0748ee84879b5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:26:40.683532 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e81a874d-9a79-4570-bad1-f1ff02e85dba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:40.685878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e81a874d-9a79-4570-bad1-f1ff02e85dba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:40.699875 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1153/2308] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:26:39 2022] DELETE /identity/v3/projects/14dfbc5cd87146368f0748ee84879b5b => generated 0 bytes in 1597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:40.700439 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c170a-3e37-468b-b77e-738264a2e69d service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:40.710485 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c170a-3e37-468b-b77e-738264a2e69d service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:40.817273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f4f86-c460-4342-acb0-c00ec31fa52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:40.818162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f4f86-c460-4342-acb0-c00ec31fa52a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:40.818970 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f4f86-c460-4342-acb0-c00ec31fa52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:40.819225 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f4f86-c460-4342-acb0-c00ec31fa52a admin admin] PATH_INFO: `/v3/users/768cb73d7b21456d97b14b1f968513de` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:40.832998 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f4f86-c460-4342-acb0-c00ec31fa52a admin admin] RBAC: Authorizing `identity:delete_user(user_id=768cb73d7b21456d97b14b1f968513de)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:40.834983 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f4f86-c460-4342-acb0-c00ec31fa52a admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:41.125550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-e16f4f86-c460-4342-acb0-c00ec31fa52a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '768cb73d7b21456d97b14b1f968513de'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:41.139853 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-e16f4f86-c460-4342-acb0-c00ec31fa52a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '768cb73d7b21456d97b14b1f968513de'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:41.144078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-e16f4f86-c460-4342-acb0-c00ec31fa52a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '768cb73d7b21456d97b14b1f968513de'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:41.149306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-e16f4f86-c460-4342-acb0-c00ec31fa52a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '768cb73d7b21456d97b14b1f968513de'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:41.166498 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-e16f4f86-c460-4342-acb0-c00ec31fa52a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '768cb73d7b21456d97b14b1f968513de'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:41.328055 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1156/2309] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:26:40 2022] DELETE /identity/v3/users/768cb73d7b21456d97b14b1f968513de => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:42.178316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a874d-9a79-4570-bad1-f1ff02e85dba admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:42.180600 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e81a874d-9a79-4570-bad1-f1ff02e85dba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:42.224577 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a874d-9a79-4570-bad1-f1ff02e85dba admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:42.340223 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe087ec-3927-4a46-bfe8-455a5bf0b599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:42.350466 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe087ec-3927-4a46-bfe8-455a5bf0b599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:42.350466 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe087ec-3927-4a46-bfe8-455a5bf0b599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:42.350466 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe087ec-3927-4a46-bfe8-455a5bf0b599 admin admin] PATH_INFO: `/v3/projects/ba19986bddee45f0a79de1d995a07872` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:42.350466 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe087ec-3927-4a46-bfe8-455a5bf0b599 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba19986bddee45f0a79de1d995a07872)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:42.350466 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe087ec-3927-4a46-bfe8-455a5bf0b599 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:42.545840 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-cbe087ec-3927-4a46-bfe8-455a5bf0b599 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba19986bddee45f0a79de1d995a07872', 'request_id': 'req-cbe087ec-3927-4a46-bfe8-455a5bf0b599'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:42.578576 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-cbe087ec-3927-4a46-bfe8-455a5bf0b599 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba19986bddee45f0a79de1d995a07872', 'request_id': 'req-cbe087ec-3927-4a46-bfe8-455a5bf0b599'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:42.578576 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-cbe087ec-3927-4a46-bfe8-455a5bf0b599 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba19986bddee45f0a79de1d995a07872', 'request_id': 'req-cbe087ec-3927-4a46-bfe8-455a5bf0b599'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:42.767791 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-cbe087ec-3927-4a46-bfe8-455a5bf0b599 admin admin] The token cache is being invalidate because project ba19986bddee45f0a79de1d995a07872 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:26:42.767791 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-cbe087ec-3927-4a46-bfe8-455a5bf0b599 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:42.767791 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-cbe087ec-3927-4a46-bfe8-455a5bf0b599 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:42.767791 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1154/2310] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:26:42 2022] DELETE /identity/v3/projects/ba19986bddee45f0a79de1d995a07872 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:42.768261 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f4f86-c460-4342-acb0-c00ec31fa52a admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:42.768261 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-e16f4f86-c460-4342-acb0-c00ec31fa52a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:42.785206 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f4f86-c460-4342-acb0-c00ec31fa52a admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:42.910352 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00f6d3-764a-48c3-ae58-26bfbc79badd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:42.911513 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00f6d3-764a-48c3-ae58-26bfbc79badd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:42.911911 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00f6d3-764a-48c3-ae58-26bfbc79badd admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:42.912289 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00f6d3-764a-48c3-ae58-26bfbc79badd admin admin] PATH_INFO: `/v3/users/d490bac4439d45bfa8453b1e238a36fe` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:42.913911 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00f6d3-764a-48c3-ae58-26bfbc79badd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d490bac4439d45bfa8453b1e238a36fe)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:42.916524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00f6d3-764a-48c3-ae58-26bfbc79badd admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:43.180368 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-ae00f6d3-764a-48c3-ae58-26bfbc79badd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd490bac4439d45bfa8453b1e238a36fe'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:43.189640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-ae00f6d3-764a-48c3-ae58-26bfbc79badd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd490bac4439d45bfa8453b1e238a36fe'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:43.190314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-ae00f6d3-764a-48c3-ae58-26bfbc79badd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd490bac4439d45bfa8453b1e238a36fe'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:43.190980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-ae00f6d3-764a-48c3-ae58-26bfbc79badd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd490bac4439d45bfa8453b1e238a36fe'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:43.225518 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-ae00f6d3-764a-48c3-ae58-26bfbc79badd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd490bac4439d45bfa8453b1e238a36fe'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:43.380553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1157/2311] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:26:42 2022] DELETE /identity/v3/users/d490bac4439d45bfa8453b1e238a36fe => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:43.428435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:43.606041 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:44.200761 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948499c-77e0-4a0b-9cdc-4299573d9640 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:44.206380 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948499c-77e0-4a0b-9cdc-4299573d9640 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:44.206380 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948499c-77e0-4a0b-9cdc-4299573d9640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:44.210273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00f6d3-764a-48c3-ae58-26bfbc79badd admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:44.212079 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ae00f6d3-764a-48c3-ae58-26bfbc79badd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:44.236113 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-1948499c-77e0-4a0b-9cdc-4299573d9640 None None] MFA Rules not processed for user `65220457c29c46358fee054fb9366406`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:44.237353 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-1948499c-77e0-4a0b-9cdc-4299573d9640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:44.283779 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00f6d3-764a-48c3-ae58-26bfbc79badd admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:44.288314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c2822-1412-4213-808b-731d3c3ca50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:44.289185 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c2822-1412-4213-808b-731d3c3ca50e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:44.289467 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c2822-1412-4213-808b-731d3c3ca50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:44.289915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c2822-1412-4213-808b-731d3c3ca50e admin admin] PATH_INFO: `/v3/projects/1753f2c03fee482fb55bb4d41654393f` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:44.296253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c2822-1412-4213-808b-731d3c3ca50e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1753f2c03fee482fb55bb4d41654393f)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:44.296253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c2822-1412-4213-808b-731d3c3ca50e admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:44.304294 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1155/2312] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:44.312062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948499c-77e0-4a0b-9cdc-4299573d9640 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:44.313901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-1948499c-77e0-4a0b-9cdc-4299573d9640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:44.328939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948499c-77e0-4a0b-9cdc-4299573d9640 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:44.335528 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e49fa2-100d-40c8-8a8d-f221eb6a3615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:44.336343 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e49fa2-100d-40c8-8a8d-f221eb6a3615 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:44.336690 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e49fa2-100d-40c8-8a8d-f221eb6a3615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:44.336978 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e49fa2-100d-40c8-8a8d-f221eb6a3615 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:44.337740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e49fa2-100d-40c8-8a8d-f221eb6a3615 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:44.339701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e49fa2-100d-40c8-8a8d-f221eb6a3615 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:44.426196 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-852c2822-1412-4213-808b-731d3c3ca50e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1753f2c03fee482fb55bb4d41654393f', 'request_id': 'req-852c2822-1412-4213-808b-731d3c3ca50e'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:44.433743 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-852c2822-1412-4213-808b-731d3c3ca50e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1753f2c03fee482fb55bb4d41654393f', 'request_id': 'req-852c2822-1412-4213-808b-731d3c3ca50e'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:44.466163 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1156/2313] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:26:44 2022] POST /identity/v3/projects => generated 371 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:26:44.470671 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e49fa2-100d-40c8-8a8d-f221eb6a3615 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:44.539939 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-852c2822-1412-4213-808b-731d3c3ca50e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1753f2c03fee482fb55bb4d41654393f', 'request_id': 'req-852c2822-1412-4213-808b-731d3c3ca50e'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:44.546713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e49fa2-100d-40c8-8a8d-f221eb6a3615 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:44.552967 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b7e26-f6bc-4d1d-8704-c67266c94598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:44.552967 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b7e26-f6bc-4d1d-8704-c67266c94598 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:44.552967 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b7e26-f6bc-4d1d-8704-c67266c94598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:44.553479 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b7e26-f6bc-4d1d-8704-c67266c94598 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:44.553574 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-852c2822-1412-4213-808b-731d3c3ca50e admin admin] The token cache is being invalidate because project 1753f2c03fee482fb55bb4d41654393f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:26:44.553855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-852c2822-1412-4213-808b-731d3c3ca50e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:44.554163 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b7e26-f6bc-4d1d-8704-c67266c94598 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:44.554731 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-852c2822-1412-4213-808b-731d3c3ca50e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:44.556280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b7e26-f6bc-4d1d-8704-c67266c94598 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:44.556598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1158/2314] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:26:44 2022] DELETE /identity/v3/projects/1753f2c03fee482fb55bb4d41654393f => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:44.562753 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355baa60-f3b2-42e8-9a4e-356707efcfbf None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:44.563315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355baa60-f3b2-42e8-9a4e-356707efcfbf None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:44.563771 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355baa60-f3b2-42e8-9a4e-356707efcfbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:44.584475 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-355baa60-f3b2-42e8-9a4e-356707efcfbf None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:44.585693 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-355baa60-f3b2-42e8-9a4e-356707efcfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:44.593762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1159/2315] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:26:44.601872 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada0abb-ba72-47f3-b793-61f3b5fe48cc None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:44.602162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada0abb-ba72-47f3-b793-61f3b5fe48cc None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:44.602439 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada0abb-ba72-47f3-b793-61f3b5fe48cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:44.620817 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-4ada0abb-ba72-47f3-b793-61f3b5fe48cc None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:44.622005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-4ada0abb-ba72-47f3-b793-61f3b5fe48cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:44.630586 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1160/2316] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:26:44.653472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada0abb-ba72-47f3-b793-61f3b5fe48cc None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:44.662818 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada0abb-ba72-47f3-b793-61f3b5fe48cc None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:44.667834 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e7745-e1aa-4f29-84c6-4ce54a8040b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:44.668496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e7745-e1aa-4f29-84c6-4ce54a8040b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:44.668779 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e7745-e1aa-4f29-84c6-4ce54a8040b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:44.669052 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e7745-e1aa-4f29-84c6-4ce54a8040b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:44.670318 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e7745-e1aa-4f29-84c6-4ce54a8040b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:26:44.670622 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e7745-e1aa-4f29-84c6-4ce54a8040b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:44.673682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e7745-e1aa-4f29-84c6-4ce54a8040b4 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:44.681241 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1157/2317] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:26:44 2022] POST /identity/v3/users => generated 292 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:26:44.685148 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b7e26-f6bc-4d1d-8704-c67266c94598 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:44.687958 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-6b9b7e26-f6bc-4d1d-8704-c67266c94598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:44.691091 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1161/2318] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:26:44.692212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e7745-e1aa-4f29-84c6-4ce54a8040b4 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:44.702227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b7e26-f6bc-4d1d-8704-c67266c94598 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:44.702596 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e7745-e1aa-4f29-84c6-4ce54a8040b4 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:44.706978 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c22c4d-4b60-42ae-be50-1082fd8b9d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:44.707287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0d18f-fd51-489e-adfe-b0e72d9831f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:44.707614 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c22c4d-4b60-42ae-be50-1082fd8b9d08 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:44.707905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c22c4d-4b60-42ae-be50-1082fd8b9d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:44.708007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0d18f-fd51-489e-adfe-b0e72d9831f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:44.708185 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c22c4d-4b60-42ae-be50-1082fd8b9d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:44.708292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0d18f-fd51-489e-adfe-b0e72d9831f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:44.708468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0d18f-fd51-489e-adfe-b0e72d9831f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:44.708919 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c22c4d-4b60-42ae-be50-1082fd8b9d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:44.709165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0d18f-fd51-489e-adfe-b0e72d9831f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:44.710517 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c22c4d-4b60-42ae-be50-1082fd8b9d08 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:44.711224 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0d18f-fd51-489e-adfe-b0e72d9831f3 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:44.724977 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1158/2319] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:26:44.725994 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c22c4d-4b60-42ae-be50-1082fd8b9d08 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:44.735898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c22c4d-4b60-42ae-be50-1082fd8b9d08 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:44.741510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d13648-edba-43db-a5e3-3ff6c60fc4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:44.742408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d13648-edba-43db-a5e3-3ff6c60fc4a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:44.742701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d13648-edba-43db-a5e3-3ff6c60fc4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:44.743024 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d13648-edba-43db-a5e3-3ff6c60fc4a0 admin admin] PATH_INFO: `/v3/projects/ef7b8d3e156e464fa3330d4376b39310/users/ec8902d03f434aa980e66acd0d159ddb/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:44.759018 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d13648-edba-43db-a5e3-3ff6c60fc4a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef7b8d3e156e464fa3330d4376b39310, user_id=ec8902d03f434aa980e66acd0d159ddb, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:44.761738 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d13648-edba-43db-a5e3-3ff6c60fc4a0 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:44.830594 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1162/2320] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:26:44 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:26:44.838194 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0d18f-fd51-489e-adfe-b0e72d9831f3 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:44.877159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1159/2321] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:26:44 2022] PUT /identity/v3/projects/ef7b8d3e156e464fa3330d4376b39310/users/ec8902d03f434aa980e66acd0d159ddb/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:44.886755 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d13648-edba-43db-a5e3-3ff6c60fc4a0 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:44.905644 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0d18f-fd51-489e-adfe-b0e72d9831f3 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:44.953161 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d13648-edba-43db-a5e3-3ff6c60fc4a0 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:44.958721 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0883f32-b78c-44c8-b96e-72636c9a8058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:44.959456 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0883f32-b78c-44c8-b96e-72636c9a8058 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:44.959741 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0883f32-b78c-44c8-b96e-72636c9a8058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:44.960180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0883f32-b78c-44c8-b96e-72636c9a8058 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:44.961180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0883f32-b78c-44c8-b96e-72636c9a8058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:44.962962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0883f32-b78c-44c8-b96e-72636c9a8058 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:44.968216 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545a352-ebf0-46ac-8a55-793e5b836a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:44.969116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545a352-ebf0-46ac-8a55-793e5b836a3a admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:44.971508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545a352-ebf0-46ac-8a55-793e5b836a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:44.971869 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545a352-ebf0-46ac-8a55-793e5b836a3a admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:44.974263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545a352-ebf0-46ac-8a55-793e5b836a3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:44.974969 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545a352-ebf0-46ac-8a55-793e5b836a3a admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:44.984377 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1160/2322] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:26:44.985408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0883f32-b78c-44c8-b96e-72636c9a8058 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:44.996068 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0883f32-b78c-44c8-b96e-72636c9a8058 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:45.000956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df1a39-7eea-44ae-ad47-21677f8cc959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:45.001985 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df1a39-7eea-44ae-ad47-21677f8cc959 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:45.002276 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df1a39-7eea-44ae-ad47-21677f8cc959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:45.002551 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df1a39-7eea-44ae-ad47-21677f8cc959 admin admin] PATH_INFO: `/v3/projects/ef7b8d3e156e464fa3330d4376b39310/users/ec8902d03f434aa980e66acd0d159ddb/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:45.004581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df1a39-7eea-44ae-ad47-21677f8cc959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef7b8d3e156e464fa3330d4376b39310, user_id=ec8902d03f434aa980e66acd0d159ddb, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:45.007248 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df1a39-7eea-44ae-ad47-21677f8cc959 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:45.019973 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1161/2323] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:26:44 2022] PUT /identity/v3/projects/ef7b8d3e156e464fa3330d4376b39310/users/ec8902d03f434aa980e66acd0d159ddb/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:45.092704 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1163/2324] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:26:44 2022] POST /identity/v3/users => generated 293 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:26:45.095153 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df1a39-7eea-44ae-ad47-21677f8cc959 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:45.198487 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df1a39-7eea-44ae-ad47-21677f8cc959 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:45.203698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920a466-212a-472b-9f52-ec28da0f882c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:45.204394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920a466-212a-472b-9f52-ec28da0f882c admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:45.204684 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920a466-212a-472b-9f52-ec28da0f882c admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:45.204963 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920a466-212a-472b-9f52-ec28da0f882c admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:45.205709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920a466-212a-472b-9f52-ec28da0f882c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:45.207388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920a466-212a-472b-9f52-ec28da0f882c admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:45.222589 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1162/2325] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:26:45.223522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545a352-ebf0-46ac-8a55-793e5b836a3a admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:45.248950 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545a352-ebf0-46ac-8a55-793e5b836a3a admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:45.260400 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d771b-1a07-4719-b904-0c33bcacfddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:45.261508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d771b-1a07-4719-b904-0c33bcacfddd admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:45.262108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d771b-1a07-4719-b904-0c33bcacfddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:45.262649 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d771b-1a07-4719-b904-0c33bcacfddd admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:45.263670 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d771b-1a07-4719-b904-0c33bcacfddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:45.265592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d771b-1a07-4719-b904-0c33bcacfddd admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:45.295098 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1164/2326] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:26:45.296546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920a466-212a-472b-9f52-ec28da0f882c admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:45.311676 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920a466-212a-472b-9f52-ec28da0f882c admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:45.323896 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc6366-1594-47e2-ba06-ec7474488f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:45.326974 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc6366-1594-47e2-ba06-ec7474488f6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:45.327391 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc6366-1594-47e2-ba06-ec7474488f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:45.328705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc6366-1594-47e2-ba06-ec7474488f6f admin admin] PATH_INFO: `/v3/projects/ac6ee29432664d98ba3d4b601272c4e3/users/7e1028d2f7574ec8b14baacf45a9d84c/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:45.353489 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc6366-1594-47e2-ba06-ec7474488f6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac6ee29432664d98ba3d4b601272c4e3, user_id=7e1028d2f7574ec8b14baacf45a9d84c, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:45.357129 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc6366-1594-47e2-ba06-ec7474488f6f admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:45.530705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1163/2327] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:26:45 2022] PUT /identity/v3/projects/ac6ee29432664d98ba3d4b601272c4e3/users/7e1028d2f7574ec8b14baacf45a9d84c/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:45.532739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea726662-fb66-4991-9a01-34fe93ad14f6 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:45.533138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea726662-fb66-4991-9a01-34fe93ad14f6 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:45.534890 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea726662-fb66-4991-9a01-34fe93ad14f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:45.620146 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-ea726662-fb66-4991-9a01-34fe93ad14f6 None None] MFA Rules not processed for user `7e1028d2f7574ec8b14baacf45a9d84c`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:45.624484 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ea726662-fb66-4991-9a01-34fe93ad14f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:45.733330 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1165/2328] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:26:45.747012 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc6366-1594-47e2-ba06-ec7474488f6f admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:45.750085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-c1fc6366-1594-47e2-ba06-ec7474488f6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:45.871320 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc6366-1594-47e2-ba06-ec7474488f6f admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:45.875533 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5cbcee-da04-4995-bdb4-961da4432bf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:45.876237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cbcee-da04-4995-bdb4-961da4432bf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:45.876529 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cbcee-da04-4995-bdb4-961da4432bf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:45.876874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cbcee-da04-4995-bdb4-961da4432bf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:45.909836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5cbcee-da04-4995-bdb4-961da4432bf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:45.912447 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5cbcee-da04-4995-bdb4-961da4432bf2 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:45.929749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1164/2329] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:46.925703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:47.305994 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:50.723411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:50.931526 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:51.770322 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fb4ba-d4e5-4d7b-a5f2-da7c8ec1ecc3 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:51.770798 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fb4ba-d4e5-4d7b-a5f2-da7c8ec1ecc3 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:51.771316 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fb4ba-d4e5-4d7b-a5f2-da7c8ec1ecc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:51.799657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-c61fb4ba-d4e5-4d7b-a5f2-da7c8ec1ecc3 None None] MFA Rules not processed for user `ec8902d03f434aa980e66acd0d159ddb`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:51.800954 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-c61fb4ba-d4e5-4d7b-a5f2-da7c8ec1ecc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:51.880681 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1166/2330] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:51 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:51.895202 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5cbcee-da04-4995-bdb4-961da4432bf2 service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:51.896902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-2b5cbcee-da04-4995-bdb4-961da4432bf2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:52.060482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5cbcee-da04-4995-bdb4-961da4432bf2 service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:52.075181 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82981a6a-b82e-4072-8ece-f61ce3aeec28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:52.076550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82981a6a-b82e-4072-8ece-f61ce3aeec28 service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:52.076843 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82981a6a-b82e-4072-8ece-f61ce3aeec28 service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:52.077128 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82981a6a-b82e-4072-8ece-f61ce3aeec28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:52.079557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-82981a6a-b82e-4072-8ece-f61ce3aeec28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:52.158389 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:52.171455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82981a6a-b82e-4072-8ece-f61ce3aeec28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:52.171455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82981a6a-b82e-4072-8ece-f61ce3aeec28 service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:52.178567 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1165/2331] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:52.205644 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:52.729022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:53.222160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db9848-b67e-465b-9ec4-3059250e0959 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:53.222725 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db9848-b67e-465b-9ec4-3059250e0959 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:53.222725 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db9848-b67e-465b-9ec4-3059250e0959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:53.255106 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-77db9848-b67e-465b-9ec4-3059250e0959 None None] MFA Rules not processed for user `484c9ed8db294be9aaeb2a7e50afabe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:53.256328 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-77db9848-b67e-465b-9ec4-3059250e0959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:53.319896 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1167/2332] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:54.429054 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:54.551690 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:55.492151 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82bbf3-ac40-408d-8917-fe1fbcb84dec None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:55.492875 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82bbf3-ac40-408d-8917-fe1fbcb84dec None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:55.493165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82bbf3-ac40-408d-8917-fe1fbcb84dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:55.512886 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-9d82bbf3-ac40-408d-8917-fe1fbcb84dec None None] MFA Rules not processed for user `65220457c29c46358fee054fb9366406`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:55.514388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-9d82bbf3-ac40-408d-8917-fe1fbcb84dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:55.524299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1166/2333] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:56.853060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:57.652496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db9848-b67e-465b-9ec4-3059250e0959 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:57.721421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db9848-b67e-465b-9ec4-3059250e0959 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:57.726986 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d7a0e-62c3-43d1-bd23-c26a8f1a2f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:57.728620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d7a0e-62c3-43d1-bd23-c26a8f1a2f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:57.734589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d7a0e-62c3-43d1-bd23-c26a8f1a2f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:57.734589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d7a0e-62c3-43d1-bd23-c26a8f1a2f76 admin admin] PATH_INFO: `/v3/users/7e1028d2f7574ec8b14baacf45a9d84c` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:57.734589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d7a0e-62c3-43d1-bd23-c26a8f1a2f76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e1028d2f7574ec8b14baacf45a9d84c)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:57.734589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d7a0e-62c3-43d1-bd23-c26a8f1a2f76 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:58.068345 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:26:58.158782 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-9f2d7a0e-62c3-43d1-bd23-c26a8f1a2f76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e1028d2f7574ec8b14baacf45a9d84c'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:58.179037 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-9f2d7a0e-62c3-43d1-bd23-c26a8f1a2f76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e1028d2f7574ec8b14baacf45a9d84c'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:58.179927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-9f2d7a0e-62c3-43d1-bd23-c26a8f1a2f76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e1028d2f7574ec8b14baacf45a9d84c'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:58.180622 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-9f2d7a0e-62c3-43d1-bd23-c26a8f1a2f76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e1028d2f7574ec8b14baacf45a9d84c'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:58.198462 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-9f2d7a0e-62c3-43d1-bd23-c26a8f1a2f76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e1028d2f7574ec8b14baacf45a9d84c'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:58.343293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1168/2334] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:26:57 2022] DELETE /identity/v3/users/7e1028d2f7574ec8b14baacf45a9d84c => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:26:58.348598 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27dac6-af00-49ae-b857-fd7b2a382fb6 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:58.349047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27dac6-af00-49ae-b857-fd7b2a382fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:58.349151 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27dac6-af00-49ae-b857-fd7b2a382fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:58.371368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-dc27dac6-af00-49ae-b857-fd7b2a382fb6 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:58.375568 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-dc27dac6-af00-49ae-b857-fd7b2a382fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:58.444925 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1167/2335] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:26:58.454394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d7a0e-62c3-43d1-bd23-c26a8f1a2f76 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:58.455840 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-9f2d7a0e-62c3-43d1-bd23-c26a8f1a2f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:58.563215 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d7a0e-62c3-43d1-bd23-c26a8f1a2f76 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:58.570900 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063cf7ac-89ba-47d8-b245-0de77c113207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:58.571600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cf7ac-89ba-47d8-b245-0de77c113207 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:58.571975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cf7ac-89ba-47d8-b245-0de77c113207 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:58.572263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cf7ac-89ba-47d8-b245-0de77c113207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:58.583955 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063cf7ac-89ba-47d8-b245-0de77c113207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:58.585440 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063cf7ac-89ba-47d8-b245-0de77c113207 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:58.603550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1169/2336] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:26:59.626737 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27dac6-af00-49ae-b857-fd7b2a382fb6 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:26:59.628484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-dc27dac6-af00-49ae-b857-fd7b2a382fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:26:59.649317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27dac6-af00-49ae-b857-fd7b2a382fb6 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:26:59.655777 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba935870-777d-4b57-8700-2de2a883e6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:26:59.662082 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba935870-777d-4b57-8700-2de2a883e6d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:59.662633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba935870-777d-4b57-8700-2de2a883e6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:59.663133 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba935870-777d-4b57-8700-2de2a883e6d0 admin admin] PATH_INFO: `/v3/projects/ac6ee29432664d98ba3d4b601272c4e3` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:59.664501 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba935870-777d-4b57-8700-2de2a883e6d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac6ee29432664d98ba3d4b601272c4e3)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:26:59.666648 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba935870-777d-4b57-8700-2de2a883e6d0 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:26:59.820768 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ba935870-777d-4b57-8700-2de2a883e6d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac6ee29432664d98ba3d4b601272c4e3', 'request_id': 'req-ba935870-777d-4b57-8700-2de2a883e6d0'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:59.828193 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ba935870-777d-4b57-8700-2de2a883e6d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac6ee29432664d98ba3d4b601272c4e3', 'request_id': 'req-ba935870-777d-4b57-8700-2de2a883e6d0'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:59.835143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ba935870-777d-4b57-8700-2de2a883e6d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac6ee29432664d98ba3d4b601272c4e3', 'request_id': 'req-ba935870-777d-4b57-8700-2de2a883e6d0'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:59.952650 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ba935870-777d-4b57-8700-2de2a883e6d0 admin admin] The token cache is being invalidate because project ac6ee29432664d98ba3d4b601272c4e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:26:59.952940 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ba935870-777d-4b57-8700-2de2a883e6d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:59.953575 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ba935870-777d-4b57-8700-2de2a883e6d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:26:59.955149 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1168/2337] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:26:59 2022] DELETE /identity/v3/projects/ac6ee29432664d98ba3d4b601272c4e3 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:26:59.960279 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3d4e6-6f9d-4ee1-8903-9ab16f043eba None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:26:59.960760 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3d4e6-6f9d-4ee1-8903-9ab16f043eba None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:26:59.960869 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3d4e6-6f9d-4ee1-8903-9ab16f043eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:26:59.979347 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-b6b3d4e6-6f9d-4ee1-8903-9ab16f043eba None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:26:59.980577 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-b6b3d4e6-6f9d-4ee1-8903-9ab16f043eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:00.039609 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1170/2338] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:27:00.047830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e846991-d89b-4167-af8c-ec7b878c5ebf None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:00.048479 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e846991-d89b-4167-af8c-ec7b878c5ebf None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:00.048772 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e846991-d89b-4167-af8c-ec7b878c5ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:00.067845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-8e846991-d89b-4167-af8c-ec7b878c5ebf None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:27:00.069059 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-8e846991-d89b-4167-af8c-ec7b878c5ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:00.077393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1169/2339] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:27:00.081750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3d4e6-6f9d-4ee1-8903-9ab16f043eba None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:00.092207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3d4e6-6f9d-4ee1-8903-9ab16f043eba None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:00.097520 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644a862-554e-4760-a327-6a2b60401233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:00.098263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644a862-554e-4760-a327-6a2b60401233 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:00.098536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644a862-554e-4760-a327-6a2b60401233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:00.098829 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644a862-554e-4760-a327-6a2b60401233 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:00.099618 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644a862-554e-4760-a327-6a2b60401233 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:27:00.099945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644a862-554e-4760-a327-6a2b60401233 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:00.101732 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644a862-554e-4760-a327-6a2b60401233 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:00.115240 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1171/2340] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:27:00.119467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e846991-d89b-4167-af8c-ec7b878c5ebf None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:00.128643 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e846991-d89b-4167-af8c-ec7b878c5ebf None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:00.133494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18da305-4053-49c7-b619-477b2319ff60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:00.134165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18da305-4053-49c7-b619-477b2319ff60 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:00.134490 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18da305-4053-49c7-b619-477b2319ff60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:00.141678 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18da305-4053-49c7-b619-477b2319ff60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:00.141678 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18da305-4053-49c7-b619-477b2319ff60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:00.141678 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18da305-4053-49c7-b619-477b2319ff60 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:00.225047 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1170/2341] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:27:00 2022] POST /identity/v3/projects => generated 359 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:27:00.226356 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644a862-554e-4760-a327-6a2b60401233 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:00.237129 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644a862-554e-4760-a327-6a2b60401233 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:00.297087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d348d-6ef2-41c5-b81b-89907f80c002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:00.297811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d348d-6ef2-41c5-b81b-89907f80c002 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:00.298099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d348d-6ef2-41c5-b81b-89907f80c002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:00.298378 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d348d-6ef2-41c5-b81b-89907f80c002 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:00.299128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d348d-6ef2-41c5-b81b-89907f80c002 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:00.301100 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d348d-6ef2-41c5-b81b-89907f80c002 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:00.307707 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:27:00.406186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1172/2342] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:27:00 2022] POST /identity/v3/users => generated 280 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:27:00.409922 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18da305-4053-49c7-b619-477b2319ff60 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:00.423379 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18da305-4053-49c7-b619-477b2319ff60 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:00.428673 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5b128-ff95-4485-9b91-5f79c8156459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:00.429405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5b128-ff95-4485-9b91-5f79c8156459 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:00.429757 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5b128-ff95-4485-9b91-5f79c8156459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:00.430041 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5b128-ff95-4485-9b91-5f79c8156459 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:00.431077 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5b128-ff95-4485-9b91-5f79c8156459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:00.432999 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5b128-ff95-4485-9b91-5f79c8156459 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:00.449101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1171/2343] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:27:00.449849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d348d-6ef2-41c5-b81b-89907f80c002 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:00.461098 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d348d-6ef2-41c5-b81b-89907f80c002 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:00.467313 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f67714-235f-4d94-aab7-e4ce581cdd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:00.468066 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f67714-235f-4d94-aab7-e4ce581cdd6e admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:00.468350 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f67714-235f-4d94-aab7-e4ce581cdd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:00.468627 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f67714-235f-4d94-aab7-e4ce581cdd6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:00.469362 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f67714-235f-4d94-aab7-e4ce581cdd6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:00.470983 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f67714-235f-4d94-aab7-e4ce581cdd6e admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:00.484425 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1173/2344] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:27:00.486281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5b128-ff95-4485-9b91-5f79c8156459 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:00.497198 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5b128-ff95-4485-9b91-5f79c8156459 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:00.502451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e217ff0-b901-4944-ac2c-4ebaa516d879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:00.505474 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e217ff0-b901-4944-ac2c-4ebaa516d879 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:00.505474 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e217ff0-b901-4944-ac2c-4ebaa516d879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:00.505474 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e217ff0-b901-4944-ac2c-4ebaa516d879 admin admin] PATH_INFO: `/v3/projects/b507b9148c0d4c1f9095676e859b57c1/users/ef004e2cb9884dad8041092d4e3372a6/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:00.521841 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e217ff0-b901-4944-ac2c-4ebaa516d879 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b507b9148c0d4c1f9095676e859b57c1, user_id=ef004e2cb9884dad8041092d4e3372a6, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:00.524725 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e217ff0-b901-4944-ac2c-4ebaa516d879 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:00.663690 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1172/2345] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:27:00 2022] PUT /identity/v3/projects/b507b9148c0d4c1f9095676e859b57c1/users/ef004e2cb9884dad8041092d4e3372a6/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:27:00.678485 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a25e3-236d-411b-ac32-15c85a8d4eb6 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:00.678785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a25e3-236d-411b-ac32-15c85a8d4eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:00.679108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a25e3-236d-411b-ac32-15c85a8d4eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:00.707306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-5d1a25e3-236d-411b-ac32-15c85a8d4eb6 None None] MFA Rules not processed for user `ef004e2cb9884dad8041092d4e3372a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:27:00.708586 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5d1a25e3-236d-411b-ac32-15c85a8d4eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:00.777960 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1174/2346] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:27:00 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:27:00.811132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e217ff0-b901-4944-ac2c-4ebaa516d879 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:00.813213 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-9e217ff0-b901-4944-ac2c-4ebaa516d879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:00.905995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e217ff0-b901-4944-ac2c-4ebaa516d879 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:00.909720 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07476144-b527-488b-bd86-8b757a11dd9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:00.910377 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07476144-b527-488b-bd86-8b757a11dd9a service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:00.910664 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07476144-b527-488b-bd86-8b757a11dd9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:00.910974 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07476144-b527-488b-bd86-8b757a11dd9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:00.920199 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07476144-b527-488b-bd86-8b757a11dd9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:00.921490 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07476144-b527-488b-bd86-8b757a11dd9a service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:00.922641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:27:00.931578 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1173/2347] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:27:00 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:27:00.947300 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a25e3-236d-411b-ac32-15c85a8d4eb6 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:01.013495 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a25e3-236d-411b-ac32-15c85a8d4eb6 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:01.018291 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b54499-8cf9-415f-8a15-29cc521259e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:01.019339 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b54499-8cf9-415f-8a15-29cc521259e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:01.019631 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b54499-8cf9-415f-8a15-29cc521259e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:01.019912 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b54499-8cf9-415f-8a15-29cc521259e7 admin admin] PATH_INFO: `/v3/users/ef004e2cb9884dad8041092d4e3372a6` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:01.021134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b54499-8cf9-415f-8a15-29cc521259e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef004e2cb9884dad8041092d4e3372a6)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:01.023120 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b54499-8cf9-415f-8a15-29cc521259e7 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:01.242114 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f7b54499-8cf9-415f-8a15-29cc521259e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef004e2cb9884dad8041092d4e3372a6'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:27:01.249479 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f7b54499-8cf9-415f-8a15-29cc521259e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef004e2cb9884dad8041092d4e3372a6'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:27:01.250181 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f7b54499-8cf9-415f-8a15-29cc521259e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef004e2cb9884dad8041092d4e3372a6'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:27:01.250847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f7b54499-8cf9-415f-8a15-29cc521259e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef004e2cb9884dad8041092d4e3372a6'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:27:01.268420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f7b54499-8cf9-415f-8a15-29cc521259e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef004e2cb9884dad8041092d4e3372a6'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:27:01.353423 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1175/2348] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:27:00 2022] DELETE /identity/v3/users/ef004e2cb9884dad8041092d4e3372a6 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:27:01.360800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cf2c3-ab4c-4a27-a4f7-fb43f920b023 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:01.361130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cf2c3-ab4c-4a27-a4f7-fb43f920b023 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:01.361415 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cf2c3-ab4c-4a27-a4f7-fb43f920b023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:01.380568 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-f86cf2c3-ab4c-4a27-a4f7-fb43f920b023 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:27:01.381688 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-f86cf2c3-ab4c-4a27-a4f7-fb43f920b023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:01.448919 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1174/2349] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:27:01.459580 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b54499-8cf9-415f-8a15-29cc521259e7 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:01.461023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-f7b54499-8cf9-415f-8a15-29cc521259e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:01.480649 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3dc3b-7dd9-4fe6-8268-14be1b6f4985 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:01.480943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3dc3b-7dd9-4fe6-8268-14be1b6f4985 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:01.481223 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3dc3b-7dd9-4fe6-8268-14be1b6f4985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:01.499896 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-4fb3dc3b-7dd9-4fe6-8268-14be1b6f4985 None None] MFA Rules not processed for user `cfbaba84402d4c31bafcd12133d74c03`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:27:01.501084 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-4fb3dc3b-7dd9-4fe6-8268-14be1b6f4985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:01.525449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b54499-8cf9-415f-8a15-29cc521259e7 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:01.530327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7b5a2-04b2-4e9b-bb70-c33e04d5c2ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:01.530903 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7b5a2-04b2-4e9b-bb70-c33e04d5c2ec service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:01.531222 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7b5a2-04b2-4e9b-bb70-c33e04d5c2ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:01.531502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7b5a2-04b2-4e9b-bb70-c33e04d5c2ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:01.541620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7b5a2-04b2-4e9b-bb70-c33e04d5c2ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:01.544619 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7b5a2-04b2-4e9b-bb70-c33e04d5c2ec service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:01.552889 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1176/2350] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:27:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:27:01.562012 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1175/2351] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jun 23 09:27:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:27:01.577385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7b5a2-04b2-4e9b-bb70-c33e04d5c2ec service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:01.578861 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a8c7b5a2-04b2-4e9b-bb70-c33e04d5c2ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:01.655900 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7b5a2-04b2-4e9b-bb70-c33e04d5c2ec service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:01.664117 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4afad5-1a55-48fb-9e52-0e4fc808f5e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b1331b4f0924bb8b137a22a33c9d130', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:01.664950 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4afad5-1a55-48fb-9e52-0e4fc808f5e1 service swift] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:01.665237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4afad5-1a55-48fb-9e52-0e4fc808f5e1 service swift] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:01.665557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4afad5-1a55-48fb-9e52-0e4fc808f5e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:01.675213 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4afad5-1a55-48fb-9e52-0e4fc808f5e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:01.676859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4afad5-1a55-48fb-9e52-0e4fc808f5e1 service swift] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:01.679750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:27:01.685942 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1177/2352] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jun 23 09:27:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 09:27:02.060650 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3dc3b-7dd9-4fe6-8268-14be1b6f4985 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:02.062115 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-4fb3dc3b-7dd9-4fe6-8268-14be1b6f4985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:02.079435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3dc3b-7dd9-4fe6-8268-14be1b6f4985 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:02.084328 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586f8f5-77f1-40b2-93fd-64b687d89d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:02.085966 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586f8f5-77f1-40b2-93fd-64b687d89d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:02.086262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586f8f5-77f1-40b2-93fd-64b687d89d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:02.086554 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586f8f5-77f1-40b2-93fd-64b687d89d01 admin admin] PATH_INFO: `/v3/projects/b507b9148c0d4c1f9095676e859b57c1` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:02.087775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d586f8f5-77f1-40b2-93fd-64b687d89d01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b507b9148c0d4c1f9095676e859b57c1)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:02.090537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d586f8f5-77f1-40b2-93fd-64b687d89d01 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:02.117461 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:27:02.257808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-d586f8f5-77f1-40b2-93fd-64b687d89d01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b507b9148c0d4c1f9095676e859b57c1', 'request_id': 'req-d586f8f5-77f1-40b2-93fd-64b687d89d01'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:27:02.267117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-d586f8f5-77f1-40b2-93fd-64b687d89d01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b507b9148c0d4c1f9095676e859b57c1', 'request_id': 'req-d586f8f5-77f1-40b2-93fd-64b687d89d01'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:27:02.274221 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-d586f8f5-77f1-40b2-93fd-64b687d89d01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b507b9148c0d4c1f9095676e859b57c1', 'request_id': 'req-d586f8f5-77f1-40b2-93fd-64b687d89d01'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:27:02.365074 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-d586f8f5-77f1-40b2-93fd-64b687d89d01 admin admin] The token cache is being invalidate because project b507b9148c0d4c1f9095676e859b57c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:27:02.365372 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-d586f8f5-77f1-40b2-93fd-64b687d89d01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:27:02.366010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-d586f8f5-77f1-40b2-93fd-64b687d89d01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:27:02.367610 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1176/2353] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:27:02 2022] DELETE /identity/v3/projects/b507b9148c0d4c1f9095676e859b57c1 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:27:02.676970 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:27:03.048391 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:27:03.085153 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:27:03.833793 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:27:04.595446 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:27:09.179321 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260b514-cf53-4478-8fe5-d6eda39d0608 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:09.179912 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260b514-cf53-4478-8fe5-d6eda39d0608 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:09.179912 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260b514-cf53-4478-8fe5-d6eda39d0608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:09.197472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-7260b514-cf53-4478-8fe5-d6eda39d0608 None None] MFA Rules not processed for user `ad5452341ccc40bc94b38369dae4f2b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:27:09.198661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-7260b514-cf53-4478-8fe5-d6eda39d0608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:09.247337 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:27:09.258005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1178/2354] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:27:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:27:13.142288 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586f8f5-77f1-40b2-93fd-64b687d89d01 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:13.143757 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-d586f8f5-77f1-40b2-93fd-64b687d89d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:13.206668 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:27:13.253864 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586f8f5-77f1-40b2-93fd-64b687d89d01 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:13.259300 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2fc9e-403d-41b5-8b27-6d217f70d457 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63b20ceca40e6bceb56eda8ae9686', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:13.261052 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2fc9e-403d-41b5-8b27-6d217f70d457 service placement] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:13.261052 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2fc9e-403d-41b5-8b27-6d217f70d457 service placement] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:13.261052 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2fc9e-403d-41b5-8b27-6d217f70d457 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:13.263207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-cab2fc9e-403d-41b5-8b27-6d217f70d457 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:13.275878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2fc9e-403d-41b5-8b27-6d217f70d457 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:13.277641 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2fc9e-403d-41b5-8b27-6d217f70d457 service placement] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:13.290315 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1177/2355] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:27:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:27:19.467694 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:27:21.523024 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efcf58-2293-4598-b65f-18713a34c5d0 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:21.523621 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efcf58-2293-4598-b65f-18713a34c5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:21.523621 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efcf58-2293-4598-b65f-18713a34c5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:21.570789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-92efcf58-2293-4598-b65f-18713a34c5d0 None None] MFA Rules not processed for user `cfbaba84402d4c31bafcd12133d74c03`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:27:21.572038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-92efcf58-2293-4598-b65f-18713a34c5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:21.637711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1179/2356] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jun 23 09:27:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:27:21.659984 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2fc9e-403d-41b5-8b27-6d217f70d457 service placement] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:21.666756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-cab2fc9e-403d-41b5-8b27-6d217f70d457 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:21.767333 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2fc9e-403d-41b5-8b27-6d217f70d457 service placement] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:21.772358 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac90ab3-d60c-430e-9ec2-406b685de842 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b1331b4f0924bb8b137a22a33c9d130', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:21.773008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac90ab3-d60c-430e-9ec2-406b685de842 service swift] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:21.773295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac90ab3-d60c-430e-9ec2-406b685de842 service swift] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:21.773569 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac90ab3-d60c-430e-9ec2-406b685de842 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:21.781724 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac90ab3-d60c-430e-9ec2-406b685de842 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:21.783406 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac90ab3-d60c-430e-9ec2-406b685de842 service swift] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:21.791542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1178/2357] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jun 23 09:27:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 09:27:31.636798 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=40 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:27:31.788148 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=60 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:27:40.165152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5dacc-7fe6-4f8c-96ef-3631856c9e76 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:40.165571 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5dacc-7fe6-4f8c-96ef-3631856c9e76 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:40.165740 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5dacc-7fe6-4f8c-96ef-3631856c9e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:40.184630 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-36f5dacc-7fe6-4f8c-96ef-3631856c9e76 None None] MFA Rules not processed for user `cfbaba84402d4c31bafcd12133d74c03`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:27:40.185845 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-36f5dacc-7fe6-4f8c-96ef-3631856c9e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:40.193128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1180/2358] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jun 23 09:27:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:27:40.209722 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac90ab3-d60c-430e-9ec2-406b685de842 service swift] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:40.220131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac90ab3-d60c-430e-9ec2-406b685de842 service swift] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:40.225119 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822dc43-6177-482c-8120-7e9f113287b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b1331b4f0924bb8b137a22a33c9d130', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:40.225755 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822dc43-6177-482c-8120-7e9f113287b4 service swift] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:40.226045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822dc43-6177-482c-8120-7e9f113287b4 service swift] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:40.226326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822dc43-6177-482c-8120-7e9f113287b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:40.235100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822dc43-6177-482c-8120-7e9f113287b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:40.236685 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822dc43-6177-482c-8120-7e9f113287b4 service swift] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:40.244847 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1179/2359] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jun 23 09:27:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 09:27:43.822521 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5dacc-7fe6-4f8c-96ef-3631856c9e76 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:43.824917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-36f5dacc-7fe6-4f8c-96ef-3631856c9e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:43.888034 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5dacc-7fe6-4f8c-96ef-3631856c9e76 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:43.892440 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62fd000-a243-45a9-8cec-f4ed7e7a0cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:43.893331 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62fd000-a243-45a9-8cec-f4ed7e7a0cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:43.893615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62fd000-a243-45a9-8cec-f4ed7e7a0cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:43.893892 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62fd000-a243-45a9-8cec-f4ed7e7a0cf5 admin admin] PATH_INFO: `/v3/users/484c9ed8db294be9aaeb2a7e50afabe0` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:43.895325 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62fd000-a243-45a9-8cec-f4ed7e7a0cf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=484c9ed8db294be9aaeb2a7e50afabe0)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:43.897139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62fd000-a243-45a9-8cec-f4ed7e7a0cf5 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:44.134925 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f62fd000-a243-45a9-8cec-f4ed7e7a0cf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '484c9ed8db294be9aaeb2a7e50afabe0'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:27:44.142431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f62fd000-a243-45a9-8cec-f4ed7e7a0cf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '484c9ed8db294be9aaeb2a7e50afabe0'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:27:44.143073 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f62fd000-a243-45a9-8cec-f4ed7e7a0cf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '484c9ed8db294be9aaeb2a7e50afabe0'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:27:44.143688 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f62fd000-a243-45a9-8cec-f4ed7e7a0cf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '484c9ed8db294be9aaeb2a7e50afabe0'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:27:44.156528 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f62fd000-a243-45a9-8cec-f4ed7e7a0cf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '484c9ed8db294be9aaeb2a7e50afabe0'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:27:44.267912 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1181/2360] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:27:43 2022] DELETE /identity/v3/users/484c9ed8db294be9aaeb2a7e50afabe0 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:27:44.500404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822dc43-6177-482c-8120-7e9f113287b4 service swift] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:44.513299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822dc43-6177-482c-8120-7e9f113287b4 service swift] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:44.586872 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe4624-3ed6-4e4f-873b-809be26a7d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:44.587887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe4624-3ed6-4e4f-873b-809be26a7d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:44.588179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe4624-3ed6-4e4f-873b-809be26a7d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:44.588460 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe4624-3ed6-4e4f-873b-809be26a7d2f admin admin] PATH_INFO: `/v3/projects/6a1a64d6d0f241739bb3dad4490182ed` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:44.589659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe4624-3ed6-4e4f-873b-809be26a7d2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a1a64d6d0f241739bb3dad4490182ed)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:44.591671 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe4624-3ed6-4e4f-873b-809be26a7d2f admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:44.732132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-8bbe4624-3ed6-4e4f-873b-809be26a7d2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a1a64d6d0f241739bb3dad4490182ed', 'request_id': 'req-8bbe4624-3ed6-4e4f-873b-809be26a7d2f'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:27:44.740333 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-8bbe4624-3ed6-4e4f-873b-809be26a7d2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a1a64d6d0f241739bb3dad4490182ed', 'request_id': 'req-8bbe4624-3ed6-4e4f-873b-809be26a7d2f'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:27:44.749091 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-8bbe4624-3ed6-4e4f-873b-809be26a7d2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a1a64d6d0f241739bb3dad4490182ed', 'request_id': 'req-8bbe4624-3ed6-4e4f-873b-809be26a7d2f'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:27:44.876045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-8bbe4624-3ed6-4e4f-873b-809be26a7d2f admin admin] The token cache is being invalidate because project 6a1a64d6d0f241739bb3dad4490182ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:27:44.881045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-8bbe4624-3ed6-4e4f-873b-809be26a7d2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:27:44.881045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-8bbe4624-3ed6-4e4f-873b-809be26a7d2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:27:44.881045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1180/2361] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:27:44 2022] DELETE /identity/v3/projects/6a1a64d6d0f241739bb3dad4490182ed => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:27:44.884924 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c863e8-deee-4b87-b485-4484f46bd938 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:44.885431 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c863e8-deee-4b87-b485-4484f46bd938 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:44.885537 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c863e8-deee-4b87-b485-4484f46bd938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:44.910851 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-e1c863e8-deee-4b87-b485-4484f46bd938 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:27:44.912133 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-e1c863e8-deee-4b87-b485-4484f46bd938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:44.977239 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1182/2362] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:27:44.986493 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754919eb-1a3f-4062-a86c-5820a06881c0 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:44.986785 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754919eb-1a3f-4062-a86c-5820a06881c0 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:44.987114 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754919eb-1a3f-4062-a86c-5820a06881c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:45.005759 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-754919eb-1a3f-4062-a86c-5820a06881c0 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:27:45.006994 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-754919eb-1a3f-4062-a86c-5820a06881c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:45.014882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1181/2363] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:27:45.030071 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c863e8-deee-4b87-b485-4484f46bd938 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:45.039945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c863e8-deee-4b87-b485-4484f46bd938 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:45.046247 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621fc060-0bc5-4c25-9264-7ebc444964dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:45.047052 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fc060-0bc5-4c25-9264-7ebc444964dc admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:45.047339 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fc060-0bc5-4c25-9264-7ebc444964dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:45.047616 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fc060-0bc5-4c25-9264-7ebc444964dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:45.048374 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621fc060-0bc5-4c25-9264-7ebc444964dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:27:45.048690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621fc060-0bc5-4c25-9264-7ebc444964dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:45.050334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621fc060-0bc5-4c25-9264-7ebc444964dc admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:45.059532 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:27:45.062641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1183/2364] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:27:45.067119 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754919eb-1a3f-4062-a86c-5820a06881c0 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:45.076629 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754919eb-1a3f-4062-a86c-5820a06881c0 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:45.081327 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd570b10-71f8-4161-ad2b-9cf8c518346f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:45.081979 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd570b10-71f8-4161-ad2b-9cf8c518346f admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:45.082265 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd570b10-71f8-4161-ad2b-9cf8c518346f admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:45.082541 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd570b10-71f8-4161-ad2b-9cf8c518346f admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:45.083274 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd570b10-71f8-4161-ad2b-9cf8c518346f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:45.085165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd570b10-71f8-4161-ad2b-9cf8c518346f admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:45.255371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1182/2365] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:27:45 2022] POST /identity/v3/projects => generated 375 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:27:45.259171 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621fc060-0bc5-4c25-9264-7ebc444964dc admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:45.269328 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621fc060-0bc5-4c25-9264-7ebc444964dc admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:45.327065 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd667e-53ec-4fa3-bc9a-746e735681d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:45.327727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd667e-53ec-4fa3-bc9a-746e735681d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:45.328010 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd667e-53ec-4fa3-bc9a-746e735681d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:45.328287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd667e-53ec-4fa3-bc9a-746e735681d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:45.328984 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd667e-53ec-4fa3-bc9a-746e735681d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:45.331092 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd667e-53ec-4fa3-bc9a-746e735681d7 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:45.459797 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1184/2366] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:27:45 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:27:45.461083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd570b10-71f8-4161-ad2b-9cf8c518346f admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:45.473644 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd570b10-71f8-4161-ad2b-9cf8c518346f admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:45.478318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12e3a1-4e3a-49fd-a43c-80f2052b9f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:45.479297 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12e3a1-4e3a-49fd-a43c-80f2052b9f15 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:45.479297 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12e3a1-4e3a-49fd-a43c-80f2052b9f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:45.479539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12e3a1-4e3a-49fd-a43c-80f2052b9f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:45.480270 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12e3a1-4e3a-49fd-a43c-80f2052b9f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:45.481909 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12e3a1-4e3a-49fd-a43c-80f2052b9f15 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:45.495142 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1183/2367] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:27:45.498560 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd667e-53ec-4fa3-bc9a-746e735681d7 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:45.509355 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd667e-53ec-4fa3-bc9a-746e735681d7 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:45.514385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cd661-57b0-4877-b82a-846dbe09947e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:45.515081 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cd661-57b0-4877-b82a-846dbe09947e admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:45.515368 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cd661-57b0-4877-b82a-846dbe09947e admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:45.515656 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cd661-57b0-4877-b82a-846dbe09947e admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:45.516379 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025cd661-57b0-4877-b82a-846dbe09947e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:45.517978 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025cd661-57b0-4877-b82a-846dbe09947e admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:45.531910 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1185/2368] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:27:45.532977 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12e3a1-4e3a-49fd-a43c-80f2052b9f15 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:45.543940 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12e3a1-4e3a-49fd-a43c-80f2052b9f15 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:45.549293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803ca3c-1cf7-4e46-8828-457cdc77506e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:45.550205 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803ca3c-1cf7-4e46-8828-457cdc77506e admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:45.550492 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803ca3c-1cf7-4e46-8828-457cdc77506e admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:45.550770 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803ca3c-1cf7-4e46-8828-457cdc77506e admin admin] PATH_INFO: `/v3/projects/e0a012cfeede49989fda04fdd2ba45ad/users/72001a6a6daf4840a2c0133e25597698/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:45.565611 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803ca3c-1cf7-4e46-8828-457cdc77506e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0a012cfeede49989fda04fdd2ba45ad, user_id=72001a6a6daf4840a2c0133e25597698, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:45.568441 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803ca3c-1cf7-4e46-8828-457cdc77506e admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:45.685786 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1184/2369] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:27:45 2022] PUT /identity/v3/projects/e0a012cfeede49989fda04fdd2ba45ad/users/72001a6a6daf4840a2c0133e25597698/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:27:45.695434 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f082a14-cd08-43d2-961a-1ea4329520d8 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:45.695729 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f082a14-cd08-43d2-961a-1ea4329520d8 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:45.696010 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f082a14-cd08-43d2-961a-1ea4329520d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:45.723872 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-7f082a14-cd08-43d2-961a-1ea4329520d8 None None] MFA Rules not processed for user `72001a6a6daf4840a2c0133e25597698`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:27:45.725098 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-7f082a14-cd08-43d2-961a-1ea4329520d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:45.796440 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1186/2370] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:27:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:27:45.807344 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803ca3c-1cf7-4e46-8828-457cdc77506e admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:45.809608 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-3803ca3c-1cf7-4e46-8828-457cdc77506e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:45.830364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:27:45.873842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803ca3c-1cf7-4e46-8828-457cdc77506e admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:45.877536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687a52e-255e-4cf7-ae94-290b107a36c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:45.878195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687a52e-255e-4cf7-ae94-290b107a36c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:45.878486 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687a52e-255e-4cf7-ae94-290b107a36c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:45.878771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687a52e-255e-4cf7-ae94-290b107a36c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:45.888044 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687a52e-255e-4cf7-ae94-290b107a36c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:45.890701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687a52e-255e-4cf7-ae94-290b107a36c8 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:45.898437 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1185/2371] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:27:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:27:45.938595 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f082a14-cd08-43d2-961a-1ea4329520d8 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:27:45.941354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-7f082a14-cd08-43d2-961a-1ea4329520d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:46.011219 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f082a14-cd08-43d2-961a-1ea4329520d8 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:27:46.016295 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b54204-fdf3-4d2d-8b89-d9bcdb801eed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:27:46.016460 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b54204-fdf3-4d2d-8b89-d9bcdb801eed service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:46.016753 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b54204-fdf3-4d2d-8b89-d9bcdb801eed service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:46.017036 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b54204-fdf3-4d2d-8b89-d9bcdb801eed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:46.019112 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-50b54204-fdf3-4d2d-8b89-d9bcdb801eed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:46.053962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:27:46.098697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b54204-fdf3-4d2d-8b89-d9bcdb801eed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:27:46.100528 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b54204-fdf3-4d2d-8b89-d9bcdb801eed service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:27:46.112654 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1187/2372] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:27:45 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:27:47.823901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfdf31-8e99-4081-8716-409629af4ea8 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:47.825867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfdf31-8e99-4081-8716-409629af4ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:47.826315 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfdf31-8e99-4081-8716-409629af4ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:47.844686 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-c4cfdf31-8e99-4081-8716-409629af4ea8 None None] MFA Rules not processed for user `72001a6a6daf4840a2c0133e25597698`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:27:47.845902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-c4cfdf31-8e99-4081-8716-409629af4ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:47.853115 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1186/2373] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:27:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:27:56.049171 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9df02-710b-4e21-855d-7841719ce8b1 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:56.049673 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9df02-710b-4e21-855d-7841719ce8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:56.049814 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9df02-710b-4e21-855d-7841719ce8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:56.066622 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-f5e9df02-710b-4e21-855d-7841719ce8b1 None None] MFA Rules not processed for user `29fe5fb181e2452388d24fd0b7aa5e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:27:56.067866 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-f5e9df02-710b-4e21-855d-7841719ce8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:56.122145 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1188/2374] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:27:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:27:57.839131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=23 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:27:58.100508 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f1205-1611-4668-8b58-92da24fa8bd2 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:27:58.100839 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f1205-1611-4668-8b58-92da24fa8bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:27:58.101128 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f1205-1611-4668-8b58-92da24fa8bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:27:58.119495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-9e7f1205-1611-4668-8b58-92da24fa8bd2 None None] MFA Rules not processed for user `30bc3dba44ed4d14a67e56aa640bf9ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:27:58.120716 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-9e7f1205-1611-4668-8b58-92da24fa8bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:27:58.184746 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1187/2375] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:27:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:28:06.115503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=44 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:28:08.177692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=25 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:28:10.965754 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48928383-b757-4c39-88dc-71543f328357 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:10.966209 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48928383-b757-4c39-88dc-71543f328357 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:10.966304 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48928383-b757-4c39-88dc-71543f328357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:10.985388 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-48928383-b757-4c39-88dc-71543f328357 None None] MFA Rules not processed for user `cfbaba84402d4c31bafcd12133d74c03`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:28:10.986610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-48928383-b757-4c39-88dc-71543f328357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:11.041009 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1189/2376] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jun 23 09:28:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:28:11.058592 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f1205-1611-4668-8b58-92da24fa8bd2 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:11.060046 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-9e7f1205-1611-4668-8b58-92da24fa8bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:11.128188 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f1205-1611-4668-8b58-92da24fa8bd2 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:11.133132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff1a85-ad1e-4e56-aaa6-c75708e5a282 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b1331b4f0924bb8b137a22a33c9d130', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:11.133776 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff1a85-ad1e-4e56-aaa6-c75708e5a282 service swift] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:11.134080 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff1a85-ad1e-4e56-aaa6-c75708e5a282 service swift] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:11.134341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff1a85-ad1e-4e56-aaa6-c75708e5a282 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:11.142752 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff1a85-ad1e-4e56-aaa6-c75708e5a282 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:11.144350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff1a85-ad1e-4e56-aaa6-c75708e5a282 service swift] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:11.152876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1188/2377] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jun 23 09:28:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 09:28:12.164179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055cccb-570e-4453-b924-fa58d73030a9 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:12.164644 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055cccb-570e-4453-b924-fa58d73030a9 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:12.164753 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055cccb-570e-4453-b924-fa58d73030a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:12.182767 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-5055cccb-570e-4453-b924-fa58d73030a9 None None] MFA Rules not processed for user `cfbaba84402d4c31bafcd12133d74c03`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:28:12.184545 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5055cccb-570e-4453-b924-fa58d73030a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:12.191905 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1190/2378] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jun 23 09:28:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:28:12.204503 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff1a85-ad1e-4e56-aaa6-c75708e5a282 service swift] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:12.215245 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff1a85-ad1e-4e56-aaa6-c75708e5a282 service swift] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:12.220380 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384b709-7b97-44a0-9bab-8904ae54f342 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b1331b4f0924bb8b137a22a33c9d130', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:12.221030 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384b709-7b97-44a0-9bab-8904ae54f342 service swift] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:12.221319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384b709-7b97-44a0-9bab-8904ae54f342 service swift] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:12.221597 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384b709-7b97-44a0-9bab-8904ae54f342 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:12.229947 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384b709-7b97-44a0-9bab-8904ae54f342 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:12.237706 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384b709-7b97-44a0-9bab-8904ae54f342 service swift] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:12.240698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1189/2379] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jun 23 09:28:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 09:28:13.063511 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de89a2-ecfa-4e37-bb24-8972e720a5fa None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:13.063803 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de89a2-ecfa-4e37-bb24-8972e720a5fa None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:13.064084 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de89a2-ecfa-4e37-bb24-8972e720a5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:13.081797 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-e3de89a2-ecfa-4e37-bb24-8972e720a5fa None None] MFA Rules not processed for user `cfbaba84402d4c31bafcd12133d74c03`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:28:13.083102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-e3de89a2-ecfa-4e37-bb24-8972e720a5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:13.090437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1191/2380] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jun 23 09:28:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:28:13.102295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384b709-7b97-44a0-9bab-8904ae54f342 service swift] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:13.113570 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384b709-7b97-44a0-9bab-8904ae54f342 service swift] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:13.118533 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52d94c-49ac-473b-93fe-a9105af8e00a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b1331b4f0924bb8b137a22a33c9d130', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:13.119246 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52d94c-49ac-473b-93fe-a9105af8e00a service swift] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:13.119558 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52d94c-49ac-473b-93fe-a9105af8e00a service swift] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:13.119836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52d94c-49ac-473b-93fe-a9105af8e00a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:13.128420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52d94c-49ac-473b-93fe-a9105af8e00a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:13.130149 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52d94c-49ac-473b-93fe-a9105af8e00a service swift] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:13.138876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1190/2381] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jun 23 09:28:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 09:28:13.741044 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a74050-a496-4a78-a11f-5f645653f35e None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:13.741926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a74050-a496-4a78-a11f-5f645653f35e None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:13.742268 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a74050-a496-4a78-a11f-5f645653f35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:13.765115 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-30a74050-a496-4a78-a11f-5f645653f35e None None] MFA Rules not processed for user `cfbaba84402d4c31bafcd12133d74c03`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:28:13.766569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-30a74050-a496-4a78-a11f-5f645653f35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:13.773701 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1192/2382] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jun 23 09:28:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:28:13.786377 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52d94c-49ac-473b-93fe-a9105af8e00a service swift] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:13.798225 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52d94c-49ac-473b-93fe-a9105af8e00a service swift] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:13.803394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7bea2-0ac0-4419-a536-71bf37fdac12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b1331b4f0924bb8b137a22a33c9d130', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:13.804048 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7bea2-0ac0-4419-a536-71bf37fdac12 service swift] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:13.804341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7bea2-0ac0-4419-a536-71bf37fdac12 service swift] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:13.804628 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7bea2-0ac0-4419-a536-71bf37fdac12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:13.812893 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7bea2-0ac0-4419-a536-71bf37fdac12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:13.814482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7bea2-0ac0-4419-a536-71bf37fdac12 service swift] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:13.822503 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1191/2383] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jun 23 09:28:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 09:28:16.453289 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a74050-a496-4a78-a11f-5f645653f35e None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:16.515281 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a74050-a496-4a78-a11f-5f645653f35e None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:16.519781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991b4b3-3269-455a-997b-563a8b82f2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:16.520649 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991b4b3-3269-455a-997b-563a8b82f2b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:16.520953 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991b4b3-3269-455a-997b-563a8b82f2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:16.521215 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991b4b3-3269-455a-997b-563a8b82f2b5 admin admin] PATH_INFO: `/v3/users/72001a6a6daf4840a2c0133e25597698` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:16.522425 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c991b4b3-3269-455a-997b-563a8b82f2b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72001a6a6daf4840a2c0133e25597698)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:16.524398 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c991b4b3-3269-455a-997b-563a8b82f2b5 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:16.749395 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-c991b4b3-3269-455a-997b-563a8b82f2b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72001a6a6daf4840a2c0133e25597698'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:28:16.756286 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-c991b4b3-3269-455a-997b-563a8b82f2b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72001a6a6daf4840a2c0133e25597698'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:28:16.757042 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-c991b4b3-3269-455a-997b-563a8b82f2b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72001a6a6daf4840a2c0133e25597698'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:28:16.757669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-c991b4b3-3269-455a-997b-563a8b82f2b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72001a6a6daf4840a2c0133e25597698'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:28:16.770099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-c991b4b3-3269-455a-997b-563a8b82f2b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72001a6a6daf4840a2c0133e25597698'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:28:16.857710 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1193/2384] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:28:16 2022] DELETE /identity/v3/users/72001a6a6daf4840a2c0133e25597698 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:28:16.863166 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9fa446-a732-42e1-9fa9-23aa0938d5ee None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:16.863680 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9fa446-a732-42e1-9fa9-23aa0938d5ee None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:16.863783 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9fa446-a732-42e1-9fa9-23aa0938d5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:16.881141 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-8c9fa446-a732-42e1-9fa9-23aa0938d5ee None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:28:16.882412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-8c9fa446-a732-42e1-9fa9-23aa0938d5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:16.941081 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1192/2385] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:28:16.951258 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991b4b3-3269-455a-997b-563a8b82f2b5 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:16.952666 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-c991b4b3-3269-455a-997b-563a8b82f2b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:16.959346 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:28:17.010680 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991b4b3-3269-455a-997b-563a8b82f2b5 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:17.014797 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afa306-b073-446b-979e-b571339854bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:17.015503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afa306-b073-446b-979e-b571339854bb service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:17.015791 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afa306-b073-446b-979e-b571339854bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:17.016072 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afa306-b073-446b-979e-b571339854bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:17.025271 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65afa306-b073-446b-979e-b571339854bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:17.026715 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65afa306-b073-446b-979e-b571339854bb service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:17.037756 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1194/2386] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:28:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:28:17.305856 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9fa446-a732-42e1-9fa9-23aa0938d5ee None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:17.307398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-8c9fa446-a732-42e1-9fa9-23aa0938d5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:17.321954 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9fa446-a732-42e1-9fa9-23aa0938d5ee None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:17.328110 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce00c5-75c0-4c70-bd74-4416c1c9499b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:17.329865 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce00c5-75c0-4c70-bd74-4416c1c9499b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:17.330208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce00c5-75c0-4c70-bd74-4416c1c9499b admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:17.330477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce00c5-75c0-4c70-bd74-4416c1c9499b admin admin] PATH_INFO: `/v3/projects/e0a012cfeede49989fda04fdd2ba45ad` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:17.332237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce00c5-75c0-4c70-bd74-4416c1c9499b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0a012cfeede49989fda04fdd2ba45ad)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:17.334133 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce00c5-75c0-4c70-bd74-4416c1c9499b admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:17.478999 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:28:17.482686 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-76ce00c5-75c0-4c70-bd74-4416c1c9499b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0a012cfeede49989fda04fdd2ba45ad', 'request_id': 'req-76ce00c5-75c0-4c70-bd74-4416c1c9499b'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:28:17.490269 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-76ce00c5-75c0-4c70-bd74-4416c1c9499b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0a012cfeede49989fda04fdd2ba45ad', 'request_id': 'req-76ce00c5-75c0-4c70-bd74-4416c1c9499b'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:28:17.497103 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-76ce00c5-75c0-4c70-bd74-4416c1c9499b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0a012cfeede49989fda04fdd2ba45ad', 'request_id': 'req-76ce00c5-75c0-4c70-bd74-4416c1c9499b'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:28:17.588101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-76ce00c5-75c0-4c70-bd74-4416c1c9499b admin admin] The token cache is being invalidate because project e0a012cfeede49989fda04fdd2ba45ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:28:17.588391 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-76ce00c5-75c0-4c70-bd74-4416c1c9499b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:28:17.603347 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-76ce00c5-75c0-4c70-bd74-4416c1c9499b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:28:17.603347 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1193/2387] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:28:17 2022] DELETE /identity/v3/projects/e0a012cfeede49989fda04fdd2ba45ad => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:28:17.607553 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d722ed-605d-416b-b0c4-c0a5c131a9d1 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:17.608068 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d722ed-605d-416b-b0c4-c0a5c131a9d1 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:17.608449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d722ed-605d-416b-b0c4-c0a5c131a9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:17.627001 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-09d722ed-605d-416b-b0c4-c0a5c131a9d1 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:28:17.628342 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-09d722ed-605d-416b-b0c4-c0a5c131a9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:17.691740 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1195/2388] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:28:17.706811 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d978bc5-6f6f-41c7-9097-fcb86c5aab35 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:17.707141 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d978bc5-6f6f-41c7-9097-fcb86c5aab35 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:17.707428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d978bc5-6f6f-41c7-9097-fcb86c5aab35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:17.729655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-8d978bc5-6f6f-41c7-9097-fcb86c5aab35 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:28:17.730953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-8d978bc5-6f6f-41c7-9097-fcb86c5aab35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:17.739735 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1194/2389] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:28:17.743939 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d722ed-605d-416b-b0c4-c0a5c131a9d1 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:17.754844 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d722ed-605d-416b-b0c4-c0a5c131a9d1 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:17.759882 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5d16d-8d35-4fd3-8891-52a33defae7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:17.760543 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5d16d-8d35-4fd3-8891-52a33defae7b admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:17.760827 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5d16d-8d35-4fd3-8891-52a33defae7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:17.761618 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5d16d-8d35-4fd3-8891-52a33defae7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:17.762370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d5d16d-8d35-4fd3-8891-52a33defae7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:28:17.762678 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d5d16d-8d35-4fd3-8891-52a33defae7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:17.764394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d5d16d-8d35-4fd3-8891-52a33defae7b admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:17.776552 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1196/2390] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:28:17.780992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d978bc5-6f6f-41c7-9097-fcb86c5aab35 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:17.790333 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d978bc5-6f6f-41c7-9097-fcb86c5aab35 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:17.795488 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13a739-7026-4a07-a685-d5ca15a0572a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:17.796224 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13a739-7026-4a07-a685-d5ca15a0572a admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:17.796516 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13a739-7026-4a07-a685-d5ca15a0572a admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:17.796826 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13a739-7026-4a07-a685-d5ca15a0572a admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:17.797514 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13a739-7026-4a07-a685-d5ca15a0572a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:17.799562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13a739-7026-4a07-a685-d5ca15a0572a admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:17.952951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1195/2391] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:28:17 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:28:17.954828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5d16d-8d35-4fd3-8891-52a33defae7b admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:17.966440 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5d16d-8d35-4fd3-8891-52a33defae7b admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:18.033425 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addfc35-1c97-4dc2-8c1c-3743e199e041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:18.034138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addfc35-1c97-4dc2-8c1c-3743e199e041 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:18.034592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addfc35-1c97-4dc2-8c1c-3743e199e041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:18.034881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addfc35-1c97-4dc2-8c1c-3743e199e041 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:18.035611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4addfc35-1c97-4dc2-8c1c-3743e199e041 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:18.037614 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4addfc35-1c97-4dc2-8c1c-3743e199e041 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:18.129287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1197/2392] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:28:17 2022] POST /identity/v3/users => generated 292 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:28:18.133000 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13a739-7026-4a07-a685-d5ca15a0572a admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:18.149548 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13a739-7026-4a07-a685-d5ca15a0572a admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:18.161491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ded94-3c1b-45c2-b028-d666a8a61ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:18.162302 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ded94-3c1b-45c2-b028-d666a8a61ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:18.162667 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ded94-3c1b-45c2-b028-d666a8a61ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:18.163054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ded94-3c1b-45c2-b028-d666a8a61ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:18.164020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ded94-3c1b-45c2-b028-d666a8a61ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:18.166017 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ded94-3c1b-45c2-b028-d666a8a61ec4 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:18.180806 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1196/2393] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:28:18.182150 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addfc35-1c97-4dc2-8c1c-3743e199e041 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:18.193316 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addfc35-1c97-4dc2-8c1c-3743e199e041 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:18.198286 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ffda4-0e7d-4090-aafa-663c93984e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:18.199782 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ffda4-0e7d-4090-aafa-663c93984e4b admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:18.200270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ffda4-0e7d-4090-aafa-663c93984e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:18.200762 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ffda4-0e7d-4090-aafa-663c93984e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:18.201704 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ffda4-0e7d-4090-aafa-663c93984e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:18.203573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ffda4-0e7d-4090-aafa-663c93984e4b admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:18.216383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1198/2394] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:28:18.220454 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ded94-3c1b-45c2-b028-d666a8a61ec4 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:18.231039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ded94-3c1b-45c2-b028-d666a8a61ec4 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:18.236789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cbd33f-6de2-4b2c-b3df-4d8694a78e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:18.237683 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbd33f-6de2-4b2c-b3df-4d8694a78e45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:18.237968 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbd33f-6de2-4b2c-b3df-4d8694a78e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:18.238247 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbd33f-6de2-4b2c-b3df-4d8694a78e45 admin admin] PATH_INFO: `/v3/projects/f17a6a1110a2462b98908ecbc67564f9/users/fc1a2967d343413d8eb72de6a49ba653/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:18.254091 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cbd33f-6de2-4b2c-b3df-4d8694a78e45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f17a6a1110a2462b98908ecbc67564f9, user_id=fc1a2967d343413d8eb72de6a49ba653, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:18.256816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cbd33f-6de2-4b2c-b3df-4d8694a78e45 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:18.388147 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1197/2395] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:28:18 2022] PUT /identity/v3/projects/f17a6a1110a2462b98908ecbc67564f9/users/fc1a2967d343413d8eb72de6a49ba653/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:28:18.393250 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e37597-8e3f-4172-a465-36fd1fe6e11c None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:18.393585 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e37597-8e3f-4172-a465-36fd1fe6e11c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:18.394020 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e37597-8e3f-4172-a465-36fd1fe6e11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:18.415738 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-37e37597-8e3f-4172-a465-36fd1fe6e11c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:28:18.416956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-37e37597-8e3f-4172-a465-36fd1fe6e11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:18.425420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:28:18.475585 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1199/2396] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:28:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:28:18.485278 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cbd33f-6de2-4b2c-b3df-4d8694a78e45 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:18.487032 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-89cbd33f-6de2-4b2c-b3df-4d8694a78e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:18.556688 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cbd33f-6de2-4b2c-b3df-4d8694a78e45 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:18.561009 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb25536-a12d-48f1-bdc5-ff7e4f226aaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:18.561490 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb25536-a12d-48f1-bdc5-ff7e4f226aaf service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:18.562519 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb25536-a12d-48f1-bdc5-ff7e4f226aaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:18.562839 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb25536-a12d-48f1-bdc5-ff7e4f226aaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:18.573516 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb25536-a12d-48f1-bdc5-ff7e4f226aaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:18.580532 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb25536-a12d-48f1-bdc5-ff7e4f226aaf service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:18.586099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1198/2397] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:28:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:28:24.391188 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc71db-eb31-4a11-8074-f063f54fa2cc None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:24.391750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc71db-eb31-4a11-8074-f063f54fa2cc None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:24.391750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc71db-eb31-4a11-8074-f063f54fa2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:24.421967 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-11fc71db-eb31-4a11-8074-f063f54fa2cc None None] MFA Rules not processed for user `fc1a2967d343413d8eb72de6a49ba653`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:28:24.423536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-11fc71db-eb31-4a11-8074-f063f54fa2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:24.501050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1200/2398] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:28:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:28:24.505581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb25536-a12d-48f1-bdc5-ff7e4f226aaf service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:24.515841 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb25536-a12d-48f1-bdc5-ff7e4f226aaf service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:24.520829 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c6caa-f2a4-419d-a413-23bbcc7e6530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:24.521582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c6caa-f2a4-419d-a413-23bbcc7e6530 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:24.521940 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c6caa-f2a4-419d-a413-23bbcc7e6530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:24.522322 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c6caa-f2a4-419d-a413-23bbcc7e6530 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:24.523167 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c6caa-f2a4-419d-a413-23bbcc7e6530 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:24.525193 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c6caa-f2a4-419d-a413-23bbcc7e6530 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:24.665214 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1199/2399] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:28:24 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:28:24.669430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc71db-eb31-4a11-8074-f063f54fa2cc None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:24.680655 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc71db-eb31-4a11-8074-f063f54fa2cc None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:24.741982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74abacf-9142-46a5-ab6b-8faae877744a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:24.742688 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74abacf-9142-46a5-ab6b-8faae877744a admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:24.742996 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74abacf-9142-46a5-ab6b-8faae877744a admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:24.743272 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74abacf-9142-46a5-ab6b-8faae877744a admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:24.743973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74abacf-9142-46a5-ab6b-8faae877744a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:24.745924 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74abacf-9142-46a5-ab6b-8faae877744a admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:24.864403 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1201/2400] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:28:24 2022] POST /identity/v3/users => generated 299 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:28:24.868418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c6caa-f2a4-419d-a413-23bbcc7e6530 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:24.879231 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c6caa-f2a4-419d-a413-23bbcc7e6530 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:24.883724 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d648b3-a614-417a-81ca-5cc7747a7a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:24.884418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d648b3-a614-417a-81ca-5cc7747a7a33 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:24.884713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d648b3-a614-417a-81ca-5cc7747a7a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:24.884996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d648b3-a614-417a-81ca-5cc7747a7a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:24.885736 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d648b3-a614-417a-81ca-5cc7747a7a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:24.887487 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d648b3-a614-417a-81ca-5cc7747a7a33 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:24.901576 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1200/2401] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:28:24.909822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74abacf-9142-46a5-ab6b-8faae877744a admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:24.920918 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74abacf-9142-46a5-ab6b-8faae877744a admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:24.926764 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82fcfa-967c-4788-9944-57e2a6731632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:24.927783 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82fcfa-967c-4788-9944-57e2a6731632 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:24.928076 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82fcfa-967c-4788-9944-57e2a6731632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:24.928565 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82fcfa-967c-4788-9944-57e2a6731632 admin admin] PATH_INFO: `/v3/projects/2b59fd48833d405689eef891b34f5da3/users/4c8a7a8d6b534755b83d35f8f33c4bd7/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:24.944115 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b82fcfa-967c-4788-9944-57e2a6731632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b59fd48833d405689eef891b34f5da3, user_id=4c8a7a8d6b534755b83d35f8f33c4bd7, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:24.946800 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b82fcfa-967c-4788-9944-57e2a6731632 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:25.024997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1202/2402] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:28:24 2022] PUT /identity/v3/projects/2b59fd48833d405689eef891b34f5da3/users/4c8a7a8d6b534755b83d35f8f33c4bd7/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:28:25.029394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d648b3-a614-417a-81ca-5cc7747a7a33 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:25.042163 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d648b3-a614-417a-81ca-5cc7747a7a33 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:25.098775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:28:25.117153 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754e4d5-a2bd-4e0a-9364-80fd727bbc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:25.117786 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754e4d5-a2bd-4e0a-9364-80fd727bbc00 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:25.118077 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754e4d5-a2bd-4e0a-9364-80fd727bbc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:25.118476 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754e4d5-a2bd-4e0a-9364-80fd727bbc00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:25.119181 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754e4d5-a2bd-4e0a-9364-80fd727bbc00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:25.122549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754e4d5-a2bd-4e0a-9364-80fd727bbc00 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:25.132470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1201/2403] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:28:25.136514 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82fcfa-967c-4788-9944-57e2a6731632 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:25.147936 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82fcfa-967c-4788-9944-57e2a6731632 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:25.152997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af454712-b2cf-46b9-92e2-089c5ca04e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:25.156456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af454712-b2cf-46b9-92e2-089c5ca04e4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:25.156456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af454712-b2cf-46b9-92e2-089c5ca04e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:25.156456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af454712-b2cf-46b9-92e2-089c5ca04e4b admin admin] PATH_INFO: `/v3/projects/2b59fd48833d405689eef891b34f5da3/users/4c8a7a8d6b534755b83d35f8f33c4bd7/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:25.157277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af454712-b2cf-46b9-92e2-089c5ca04e4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b59fd48833d405689eef891b34f5da3, user_id=4c8a7a8d6b534755b83d35f8f33c4bd7, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:25.159410 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af454712-b2cf-46b9-92e2-089c5ca04e4b admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:25.163388 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:28:25.173337 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1203/2404] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:28:25 2022] PUT /identity/v3/projects/2b59fd48833d405689eef891b34f5da3/users/4c8a7a8d6b534755b83d35f8f33c4bd7/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:28:32.003827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a23ec-f6d7-4cc9-ab3f-424be287682c None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:32.004373 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a23ec-f6d7-4cc9-ab3f-424be287682c None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:32.004373 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a23ec-f6d7-4cc9-ab3f-424be287682c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:32.050806 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-811a23ec-f6d7-4cc9-ab3f-424be287682c None None] MFA Rules not processed for user `4c8a7a8d6b534755b83d35f8f33c4bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:28:32.052446 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-811a23ec-f6d7-4cc9-ab3f-424be287682c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:32.132761 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1202/2405] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:28:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:28:32.145845 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af454712-b2cf-46b9-92e2-089c5ca04e4b admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:32.148294 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-af454712-b2cf-46b9-92e2-089c5ca04e4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:32.249125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af454712-b2cf-46b9-92e2-089c5ca04e4b admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:32.252934 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a313e-9e8e-4a83-8148-0c3a867b2eac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36a6a3190dad4948a505bbe4c7095699', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:32.253597 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a313e-9e8e-4a83-8148-0c3a867b2eac service glance] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:32.253888 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a313e-9e8e-4a83-8148-0c3a867b2eac service glance] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:32.254172 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a313e-9e8e-4a83-8148-0c3a867b2eac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:32.324864 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a313e-9e8e-4a83-8148-0c3a867b2eac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:32.326638 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a313e-9e8e-4a83-8148-0c3a867b2eac service glance] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:32.336479 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1204/2406] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:28:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:28:32.614195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71b166-1853-48d2-a3fc-111320f73bc5 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:32.614494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71b166-1853-48d2-a3fc-111320f73bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:32.614776 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71b166-1853-48d2-a3fc-111320f73bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:32.634101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-2a71b166-1853-48d2-a3fc-111320f73bc5 None None] MFA Rules not processed for user `cfbaba84402d4c31bafcd12133d74c03`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:28:32.635355 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-2a71b166-1853-48d2-a3fc-111320f73bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:32.695575 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1203/2407] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jun 23 09:28:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:28:32.717907 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a313e-9e8e-4a83-8148-0c3a867b2eac service glance] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:32.719395 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-7b1a313e-9e8e-4a83-8148-0c3a867b2eac service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:32.788856 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a313e-9e8e-4a83-8148-0c3a867b2eac service glance] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:32.794403 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0c5a0-a10f-49bb-8f48-f51a59179257 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b1331b4f0924bb8b137a22a33c9d130', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:32.795083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0c5a0-a10f-49bb-8f48-f51a59179257 service swift] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:32.795389 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0c5a0-a10f-49bb-8f48-f51a59179257 service swift] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:32.795663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0c5a0-a10f-49bb-8f48-f51a59179257 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:32.805224 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0c5a0-a10f-49bb-8f48-f51a59179257 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:32.806883 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0c5a0-a10f-49bb-8f48-f51a59179257 service swift] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:32.816196 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1205/2408] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jun 23 09:28:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 09:28:34.768039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e64a5-d44b-4102-80c0-cf6275740b98 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:34.768597 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e64a5-d44b-4102-80c0-cf6275740b98 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:34.768746 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e64a5-d44b-4102-80c0-cf6275740b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:34.794851 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-0d0e64a5-d44b-4102-80c0-cf6275740b98 None None] MFA Rules not processed for user `ad5452341ccc40bc94b38369dae4f2b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:28:34.796246 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-0d0e64a5-d44b-4102-80c0-cf6275740b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:34.890534 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1204/2409] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:28:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:28:42.812626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=91 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:28:44.266817 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d22f49-d86b-4dd6-85ce-55bb1607a52f None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:44.267457 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d22f49-d86b-4dd6-85ce-55bb1607a52f None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:44.267457 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d22f49-d86b-4dd6-85ce-55bb1607a52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:44.284031 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-f1d22f49-d86b-4dd6-85ce-55bb1607a52f None None] MFA Rules not processed for user `fc1a2967d343413d8eb72de6a49ba653`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:28:44.285220 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-f1d22f49-d86b-4dd6-85ce-55bb1607a52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:44.292132 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1206/2410] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:28:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 26 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:28:44.866172 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=95 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:28:51.413739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c3173-21bd-4c1d-ab8d-fbcea59d78ce None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:51.414338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c3173-21bd-4c1d-ab8d-fbcea59d78ce None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:51.414338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c3173-21bd-4c1d-ab8d-fbcea59d78ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:51.435529 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-5a2c3173-21bd-4c1d-ab8d-fbcea59d78ce None None] MFA Rules not processed for user `cfbaba84402d4c31bafcd12133d74c03`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:28:51.437171 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-5a2c3173-21bd-4c1d-ab8d-fbcea59d78ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:51.445995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1205/2411] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jun 23 09:28:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:28:51.464408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d22f49-d86b-4dd6-85ce-55bb1607a52f None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:51.486785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d22f49-d86b-4dd6-85ce-55bb1607a52f None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:51.502698 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383b82c-25c5-4fc2-b954-775dfc81a63c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b1331b4f0924bb8b137a22a33c9d130', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:51.503661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383b82c-25c5-4fc2-b954-775dfc81a63c service swift] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:51.504128 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383b82c-25c5-4fc2-b954-775dfc81a63c service swift] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:51.504583 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383b82c-25c5-4fc2-b954-775dfc81a63c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:51.524642 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383b82c-25c5-4fc2-b954-775dfc81a63c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:51.527767 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383b82c-25c5-4fc2-b954-775dfc81a63c service swift] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:51.543927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1207/2412] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jun 23 09:28:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 09:28:53.042585 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c3173-21bd-4c1d-ab8d-fbcea59d78ce None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:53.130827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c3173-21bd-4c1d-ab8d-fbcea59d78ce None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:53.136675 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35308022-9deb-4ba2-b1b1-612e85fcb4d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:53.138118 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35308022-9deb-4ba2-b1b1-612e85fcb4d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:53.139028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35308022-9deb-4ba2-b1b1-612e85fcb4d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:53.139469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35308022-9deb-4ba2-b1b1-612e85fcb4d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:53.142143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-35308022-9deb-4ba2-b1b1-612e85fcb4d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:53.247028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35308022-9deb-4ba2-b1b1-612e85fcb4d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:53.248787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35308022-9deb-4ba2-b1b1-612e85fcb4d3 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:53.280674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1206/2413] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:28:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:28:55.758943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383b82c-25c5-4fc2-b954-775dfc81a63c service swift] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:28:55.761198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-9383b82c-25c5-4fc2-b954-775dfc81a63c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:55.773873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383b82c-25c5-4fc2-b954-775dfc81a63c service swift] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:28:55.777521 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d345bd-b9b5-4027-ac02-df8cc4ce3cac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:28:55.778312 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d345bd-b9b5-4027-ac02-df8cc4ce3cac service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:28:55.778607 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d345bd-b9b5-4027-ac02-df8cc4ce3cac service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:28:55.778889 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d345bd-b9b5-4027-ac02-df8cc4ce3cac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:28:55.780711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-93d345bd-b9b5-4027-ac02-df8cc4ce3cac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:28:55.810238 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d345bd-b9b5-4027-ac02-df8cc4ce3cac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:28:55.812058 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d345bd-b9b5-4027-ac02-df8cc4ce3cac service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:28:55.824982 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1208/2414] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:28:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:29:01.468020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35308022-9deb-4ba2-b1b1-612e85fcb4d3 service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:01.469390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-35308022-9deb-4ba2-b1b1-612e85fcb4d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:01.483734 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35308022-9deb-4ba2-b1b1-612e85fcb4d3 service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:01.489870 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c9474-ea2c-4f25-9d5c-3ec820cb425c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:01.490528 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c9474-ea2c-4f25-9d5c-3ec820cb425c service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:01.490800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c9474-ea2c-4f25-9d5c-3ec820cb425c service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:01.491099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c9474-ea2c-4f25-9d5c-3ec820cb425c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:01.492814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-d78c9474-ea2c-4f25-9d5c-3ec820cb425c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:01.553689 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c9474-ea2c-4f25-9d5c-3ec820cb425c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:01.555322 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c9474-ea2c-4f25-9d5c-3ec820cb425c service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:01.564668 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1207/2415] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:29:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:29:05.435440 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d345bd-b9b5-4027-ac02-df8cc4ce3cac service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:05.446449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d345bd-b9b5-4027-ac02-df8cc4ce3cac service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:05.451955 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0ec8f-e6e0-4daa-bfd4-ba60def938e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:05.452589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0ec8f-e6e0-4daa-bfd4-ba60def938e2 service nova] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:05.452881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0ec8f-e6e0-4daa-bfd4-ba60def938e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:05.453162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0ec8f-e6e0-4daa-bfd4-ba60def938e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:05.454977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-6be0ec8f-e6e0-4daa-bfd4-ba60def938e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:05.514313 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0ec8f-e6e0-4daa-bfd4-ba60def938e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:05.515955 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0ec8f-e6e0-4daa-bfd4-ba60def938e2 service nova] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:05.526369 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1209/2416] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:29:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:29:11.560396 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=85 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:29:15.079939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c9474-ea2c-4f25-9d5c-3ec820cb425c service nova] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:15.089495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c9474-ea2c-4f25-9d5c-3ec820cb425c service nova] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:15.094299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fec3bb-2151-4eb1-b945-4b280f2faed6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:15.094299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fec3bb-2151-4eb1-b945-4b280f2faed6 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:15.094299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fec3bb-2151-4eb1-b945-4b280f2faed6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:15.094753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fec3bb-2151-4eb1-b945-4b280f2faed6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:15.096450 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-05fec3bb-2151-4eb1-b945-4b280f2faed6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:15.186487 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fec3bb-2151-4eb1-b945-4b280f2faed6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:15.187949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fec3bb-2151-4eb1-b945-4b280f2faed6 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:15.198438 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1208/2417] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:29:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:29:15.521026 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=43 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:29:16.870974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634427c5-cc33-4ac3-9ff6-d7c50f6c20b7 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:16.871559 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634427c5-cc33-4ac3-9ff6-d7c50f6c20b7 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:16.871559 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634427c5-cc33-4ac3-9ff6-d7c50f6c20b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:16.889448 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-634427c5-cc33-4ac3-9ff6-d7c50f6c20b7 None None] MFA Rules not processed for user `cfbaba84402d4c31bafcd12133d74c03`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:29:16.890651 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-634427c5-cc33-4ac3-9ff6-d7c50f6c20b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:16.900591 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1210/2418] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jun 23 09:29:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:29:16.913355 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fec3bb-2151-4eb1-b945-4b280f2faed6 service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:16.926383 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fec3bb-2151-4eb1-b945-4b280f2faed6 service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:16.933081 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286529cf-eb7a-4ab8-995d-536d0ec8c192 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b1331b4f0924bb8b137a22a33c9d130', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:16.933998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286529cf-eb7a-4ab8-995d-536d0ec8c192 service swift] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:16.934244 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286529cf-eb7a-4ab8-995d-536d0ec8c192 service swift] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:16.934707 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286529cf-eb7a-4ab8-995d-536d0ec8c192 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:16.956107 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286529cf-eb7a-4ab8-995d-536d0ec8c192 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:16.957741 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286529cf-eb7a-4ab8-995d-536d0ec8c192 service swift] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:16.976404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1209/2419] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jun 23 09:29:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 09:29:20.076005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634427c5-cc33-4ac3-9ff6-d7c50f6c20b7 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:20.078717 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-634427c5-cc33-4ac3-9ff6-d7c50f6c20b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:20.101600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634427c5-cc33-4ac3-9ff6-d7c50f6c20b7 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:20.106833 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294eb6d4-62c2-46b6-a70e-14910c4241d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:20.107738 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294eb6d4-62c2-46b6-a70e-14910c4241d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:20.108193 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294eb6d4-62c2-46b6-a70e-14910c4241d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:20.108649 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294eb6d4-62c2-46b6-a70e-14910c4241d8 admin admin] PATH_INFO: `/v3/users/29fe5fb181e2452388d24fd0b7aa5e9f` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:20.110221 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294eb6d4-62c2-46b6-a70e-14910c4241d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29fe5fb181e2452388d24fd0b7aa5e9f)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:20.112727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294eb6d4-62c2-46b6-a70e-14910c4241d8 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:20.339314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-294eb6d4-62c2-46b6-a70e-14910c4241d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29fe5fb181e2452388d24fd0b7aa5e9f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:29:20.348027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-294eb6d4-62c2-46b6-a70e-14910c4241d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29fe5fb181e2452388d24fd0b7aa5e9f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:29:20.348784 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-294eb6d4-62c2-46b6-a70e-14910c4241d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29fe5fb181e2452388d24fd0b7aa5e9f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:29:20.349402 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-294eb6d4-62c2-46b6-a70e-14910c4241d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29fe5fb181e2452388d24fd0b7aa5e9f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:29:20.363668 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-294eb6d4-62c2-46b6-a70e-14910c4241d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29fe5fb181e2452388d24fd0b7aa5e9f'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:29:20.786246 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1211/2420] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:29:20 2022] DELETE /identity/v3/users/29fe5fb181e2452388d24fd0b7aa5e9f => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:29:21.046766 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286529cf-eb7a-4ab8-995d-536d0ec8c192 service swift] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:21.067227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286529cf-eb7a-4ab8-995d-536d0ec8c192 service swift] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:21.149018 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047e927-9b91-48d1-9ab8-4191b8c50db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:21.149999 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047e927-9b91-48d1-9ab8-4191b8c50db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:21.150263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047e927-9b91-48d1-9ab8-4191b8c50db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:21.151199 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047e927-9b91-48d1-9ab8-4191b8c50db0 admin admin] PATH_INFO: `/v3/projects/368a6f8fbca64e7791d5330f0a99d086` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:21.153104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c047e927-9b91-48d1-9ab8-4191b8c50db0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=368a6f8fbca64e7791d5330f0a99d086)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:21.160239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c047e927-9b91-48d1-9ab8-4191b8c50db0 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:21.295705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c047e927-9b91-48d1-9ab8-4191b8c50db0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '368a6f8fbca64e7791d5330f0a99d086', 'request_id': 'req-c047e927-9b91-48d1-9ab8-4191b8c50db0'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:29:21.303179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c047e927-9b91-48d1-9ab8-4191b8c50db0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '368a6f8fbca64e7791d5330f0a99d086', 'request_id': 'req-c047e927-9b91-48d1-9ab8-4191b8c50db0'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:29:21.310251 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c047e927-9b91-48d1-9ab8-4191b8c50db0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '368a6f8fbca64e7791d5330f0a99d086', 'request_id': 'req-c047e927-9b91-48d1-9ab8-4191b8c50db0'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:29:21.428002 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c047e927-9b91-48d1-9ab8-4191b8c50db0 admin admin] The token cache is being invalidate because project 368a6f8fbca64e7791d5330f0a99d086 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:29:21.428250 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c047e927-9b91-48d1-9ab8-4191b8c50db0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:29:21.428844 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c047e927-9b91-48d1-9ab8-4191b8c50db0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:29:21.430435 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1210/2421] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:29:21 2022] DELETE /identity/v3/projects/368a6f8fbca64e7791d5330f0a99d086 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:29:21.435600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b0a04-e3d9-478e-920c-df34232e75d1 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:21.436130 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b0a04-e3d9-478e-920c-df34232e75d1 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:21.436243 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b0a04-e3d9-478e-920c-df34232e75d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:21.458238 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-3a6b0a04-e3d9-478e-920c-df34232e75d1 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:29:21.460111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-3a6b0a04-e3d9-478e-920c-df34232e75d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:21.539315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1212/2422] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:29:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:29:21.539961 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919843d-f8eb-4696-8e34-7b5b365f5f6c None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:21.540256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919843d-f8eb-4696-8e34-7b5b365f5f6c None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:21.540537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919843d-f8eb-4696-8e34-7b5b365f5f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:21.560646 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-5919843d-f8eb-4696-8e34-7b5b365f5f6c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:29:21.562024 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-5919843d-f8eb-4696-8e34-7b5b365f5f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:21.570103 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1211/2423] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:29:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:29:21.575676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b0a04-e3d9-478e-920c-df34232e75d1 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:21.586038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b0a04-e3d9-478e-920c-df34232e75d1 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:21.590754 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9b9bd-67cb-44dc-abc5-3fe9d084077a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:21.593876 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9b9bd-67cb-44dc-abc5-3fe9d084077a admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:21.594163 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9b9bd-67cb-44dc-abc5-3fe9d084077a admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:21.594448 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9b9bd-67cb-44dc-abc5-3fe9d084077a admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:21.595216 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9b9bd-67cb-44dc-abc5-3fe9d084077a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:29:21.595525 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9b9bd-67cb-44dc-abc5-3fe9d084077a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:21.597337 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9b9bd-67cb-44dc-abc5-3fe9d084077a admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:21.613961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1213/2424] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:29:21.618336 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919843d-f8eb-4696-8e34-7b5b365f5f6c None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:21.625486 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:29:21.631160 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919843d-f8eb-4696-8e34-7b5b365f5f6c None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:21.636778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4418d6a-1b13-4f21-b7e1-8df4f0599722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:21.638351 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4418d6a-1b13-4f21-b7e1-8df4f0599722 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:21.638742 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4418d6a-1b13-4f21-b7e1-8df4f0599722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:21.639158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4418d6a-1b13-4f21-b7e1-8df4f0599722 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:21.640008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4418d6a-1b13-4f21-b7e1-8df4f0599722 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:21.642409 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4418d6a-1b13-4f21-b7e1-8df4f0599722 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:21.748649 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1212/2425] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:29:21 2022] POST /identity/v3/projects => generated 375 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:29:21.755580 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9b9bd-67cb-44dc-abc5-3fe9d084077a admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:21.771126 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9b9bd-67cb-44dc-abc5-3fe9d084077a admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:21.832663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:29:21.841461 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a3c16-d8f6-4bec-9a67-ecc585a34115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:21.842074 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a3c16-d8f6-4bec-9a67-ecc585a34115 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:21.845202 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a3c16-d8f6-4bec-9a67-ecc585a34115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:21.845202 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a3c16-d8f6-4bec-9a67-ecc585a34115 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:21.845202 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a3c16-d8f6-4bec-9a67-ecc585a34115 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:21.845490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a3c16-d8f6-4bec-9a67-ecc585a34115 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:21.991889 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1214/2426] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:29:21 2022] POST /identity/v3/users => generated 288 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:29:21.996457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4418d6a-1b13-4f21-b7e1-8df4f0599722 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:22.007345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4418d6a-1b13-4f21-b7e1-8df4f0599722 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:22.016295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcb9b9-6ab9-4c3b-aaf7-38b0689510e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:22.016971 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcb9b9-6ab9-4c3b-aaf7-38b0689510e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:22.017264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcb9b9-6ab9-4c3b-aaf7-38b0689510e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:22.017542 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcb9b9-6ab9-4c3b-aaf7-38b0689510e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:22.018272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcb9b9-6ab9-4c3b-aaf7-38b0689510e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:22.019904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcb9b9-6ab9-4c3b-aaf7-38b0689510e0 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:22.032341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1213/2427] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:29:22.036826 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a3c16-d8f6-4bec-9a67-ecc585a34115 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:22.049139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a3c16-d8f6-4bec-9a67-ecc585a34115 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:22.065815 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2a372-27fd-4195-9f1a-c99aeb3db793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:22.066539 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2a372-27fd-4195-9f1a-c99aeb3db793 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:22.066832 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2a372-27fd-4195-9f1a-c99aeb3db793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:22.071118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2a372-27fd-4195-9f1a-c99aeb3db793 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:22.071924 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2a372-27fd-4195-9f1a-c99aeb3db793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:22.074092 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2a372-27fd-4195-9f1a-c99aeb3db793 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:22.101370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1215/2428] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:29:22.105954 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcb9b9-6ab9-4c3b-aaf7-38b0689510e0 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:22.118562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcb9b9-6ab9-4c3b-aaf7-38b0689510e0 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:22.124042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd16613-c4a4-4f4c-aa11-ce43c74df72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:22.125604 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd16613-c4a4-4f4c-aa11-ce43c74df72d admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:22.126891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd16613-c4a4-4f4c-aa11-ce43c74df72d admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:22.127441 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd16613-c4a4-4f4c-aa11-ce43c74df72d admin admin] PATH_INFO: `/v3/projects/19a424690e5b44d58c27c5f872c38bd4/users/9f0d4416e74a4126a6aa1ccf58e22ee3/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:22.146332 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd16613-c4a4-4f4c-aa11-ce43c74df72d admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a424690e5b44d58c27c5f872c38bd4, user_id=9f0d4416e74a4126a6aa1ccf58e22ee3, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:22.151950 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd16613-c4a4-4f4c-aa11-ce43c74df72d admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:22.236039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1214/2429] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:29:22 2022] PUT /identity/v3/projects/19a424690e5b44d58c27c5f872c38bd4/users/9f0d4416e74a4126a6aa1ccf58e22ee3/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:29:22.241050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64435c-6f14-4677-989f-80dc237dee9d None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:22.241439 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64435c-6f14-4677-989f-80dc237dee9d None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:22.241824 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64435c-6f14-4677-989f-80dc237dee9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:22.269374 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-2c64435c-6f14-4677-989f-80dc237dee9d None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:29:22.270539 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-2c64435c-6f14-4677-989f-80dc237dee9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:22.355689 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1216/2430] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:29:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:29:22.365904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd16613-c4a4-4f4c-aa11-ce43c74df72d admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:22.367755 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-2fd16613-c4a4-4f4c-aa11-ce43c74df72d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:22.442732 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd16613-c4a4-4f4c-aa11-ce43c74df72d admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:22.446562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40fc7c-d2fd-4451-bdcb-94e7bdc3ebbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:22.447263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40fc7c-d2fd-4451-bdcb-94e7bdc3ebbd service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:22.448101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40fc7c-d2fd-4451-bdcb-94e7bdc3ebbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:22.448379 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40fc7c-d2fd-4451-bdcb-94e7bdc3ebbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:22.459902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd40fc7c-d2fd-4451-bdcb-94e7bdc3ebbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:22.461349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd40fc7c-d2fd-4451-bdcb-94e7bdc3ebbd service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:22.474068 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1215/2431] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:29:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:29:26.379857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64435c-6f14-4677-989f-80dc237dee9d None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:26.391559 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64435c-6f14-4677-989f-80dc237dee9d None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:26.405754 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e058493-3722-4ec3-a72c-8e0021e1670d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:26.406771 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e058493-3722-4ec3-a72c-8e0021e1670d service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:26.407500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e058493-3722-4ec3-a72c-8e0021e1670d service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:26.413143 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e058493-3722-4ec3-a72c-8e0021e1670d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:26.415811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-2e058493-3722-4ec3-a72c-8e0021e1670d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:26.542579 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e058493-3722-4ec3-a72c-8e0021e1670d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:26.544771 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e058493-3722-4ec3-a72c-8e0021e1670d service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:26.568669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1217/2432] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:29:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:29:29.901735 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40fc7c-d2fd-4451-bdcb-94e7bdc3ebbd service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:29.901735 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-bd40fc7c-d2fd-4451-bdcb-94e7bdc3ebbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:30.074069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40fc7c-d2fd-4451-bdcb-94e7bdc3ebbd service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:30.085275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed4596-f5f0-4a79-9fb2-35af6ace70d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:30.085963 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed4596-f5f0-4a79-9fb2-35af6ace70d7 service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:30.086253 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed4596-f5f0-4a79-9fb2-35af6ace70d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:30.086533 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed4596-f5f0-4a79-9fb2-35af6ace70d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:30.092019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e6ed4596-f5f0-4a79-9fb2-35af6ace70d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:30.107881 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed4596-f5f0-4a79-9fb2-35af6ace70d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:30.119767 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed4596-f5f0-4a79-9fb2-35af6ace70d7 service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:30.127427 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1216/2433] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:29:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:29:30.305948 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e058493-3722-4ec3-a72c-8e0021e1670d service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:30.308057 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-2e058493-3722-4ec3-a72c-8e0021e1670d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:30.334472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e058493-3722-4ec3-a72c-8e0021e1670d service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:30.364706 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea980963-d1b9-4e16-aaa5-bd171958585f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:30.365967 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea980963-d1b9-4e16-aaa5-bd171958585f service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:30.366413 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea980963-d1b9-4e16-aaa5-bd171958585f service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:30.366845 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea980963-d1b9-4e16-aaa5-bd171958585f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:30.369456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ea980963-d1b9-4e16-aaa5-bd171958585f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:30.388789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea980963-d1b9-4e16-aaa5-bd171958585f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:30.390684 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea980963-d1b9-4e16-aaa5-bd171958585f service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:30.410133 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1218/2434] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:29:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:29:30.552329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac13724-0fa9-4926-bd84-53cc5203c6e9 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:30.552671 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac13724-0fa9-4926-bd84-53cc5203c6e9 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:30.552975 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac13724-0fa9-4926-bd84-53cc5203c6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:30.604317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-5ac13724-0fa9-4926-bd84-53cc5203c6e9 None None] MFA Rules not processed for user `9f0d4416e74a4126a6aa1ccf58e22ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:29:30.610386 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-5ac13724-0fa9-4926-bd84-53cc5203c6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:30.640317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:29:30.693257 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1217/2435] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:29:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:29:30.708667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea980963-d1b9-4e16-aaa5-bd171958585f service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:30.724310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea980963-d1b9-4e16-aaa5-bd171958585f service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:30.744839 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122683d-75d4-4684-b863-579114f9fca7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:30.745835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122683d-75d4-4684-b863-579114f9fca7 service nova] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:30.746836 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122683d-75d4-4684-b863-579114f9fca7 service nova] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:30.747221 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122683d-75d4-4684-b863-579114f9fca7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:30.784968 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122683d-75d4-4684-b863-579114f9fca7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:30.787270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122683d-75d4-4684-b863-579114f9fca7 service nova] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:30.814424 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1219/2436] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:29:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:29:35.128706 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924098f-da6a-471e-9699-69d6a2a136c2 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:35.129235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924098f-da6a-471e-9699-69d6a2a136c2 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:35.129335 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924098f-da6a-471e-9699-69d6a2a136c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:35.153937 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-b924098f-da6a-471e-9699-69d6a2a136c2 None None] MFA Rules not processed for user `9f0d4416e74a4126a6aa1ccf58e22ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:29:35.155719 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b924098f-da6a-471e-9699-69d6a2a136c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:35.164467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1218/2437] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:29:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:29:35.569575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122683d-75d4-4684-b863-579114f9fca7 service nova] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:35.579475 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924098f-da6a-471e-9699-69d6a2a136c2 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:35.581206 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b924098f-da6a-471e-9699-69d6a2a136c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:35.583331 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122683d-75d4-4684-b863-579114f9fca7 service nova] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:35.588999 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26619e-f99e-42e0-a585-2bd9fbb98ff0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:35.589806 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26619e-f99e-42e0-a585-2bd9fbb98ff0 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:35.590198 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26619e-f99e-42e0-a585-2bd9fbb98ff0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:35.590624 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26619e-f99e-42e0-a585-2bd9fbb98ff0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:35.593232 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-9c26619e-f99e-42e0-a585-2bd9fbb98ff0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:35.602158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924098f-da6a-471e-9699-69d6a2a136c2 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:35.606490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c26619e-f99e-42e0-a585-2bd9fbb98ff0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:35.608097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c26619e-f99e-42e0-a585-2bd9fbb98ff0 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:35.610295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e613387-e596-41b9-a33c-df893e735ff7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:35.611575 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e613387-e596-41b9-a33c-df893e735ff7 service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:35.612207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e613387-e596-41b9-a33c-df893e735ff7 service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:35.612603 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e613387-e596-41b9-a33c-df893e735ff7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:35.616335 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7e613387-e596-41b9-a33c-df893e735ff7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:35.617180 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1220/2438] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:29:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:29:35.641089 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e613387-e596-41b9-a33c-df893e735ff7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:35.644798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e613387-e596-41b9-a33c-df893e735ff7 service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:35.679148 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1219/2439] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:29:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:29:36.208472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26619e-f99e-42e0-a585-2bd9fbb98ff0 service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:36.210138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-9c26619e-f99e-42e0-a585-2bd9fbb98ff0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:36.297571 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:29:36.361131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26619e-f99e-42e0-a585-2bd9fbb98ff0 service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:36.366773 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7274b-d7cf-4030-9c42-2dfdac78b9cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63b20ceca40e6bceb56eda8ae9686', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:36.367520 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7274b-d7cf-4030-9c42-2dfdac78b9cb service placement] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:36.368301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7274b-d7cf-4030-9c42-2dfdac78b9cb service placement] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:36.368647 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7274b-d7cf-4030-9c42-2dfdac78b9cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:36.371046 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-c7e7274b-d7cf-4030-9c42-2dfdac78b9cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:36.383998 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7274b-d7cf-4030-9c42-2dfdac78b9cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:36.386164 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7274b-d7cf-4030-9c42-2dfdac78b9cb service placement] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:36.396600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1221/2440] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:29:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:29:38.674358 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e613387-e596-41b9-a33c-df893e735ff7 service nova] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:38.683749 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e613387-e596-41b9-a33c-df893e735ff7 service nova] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:38.687396 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e512ce-bebd-46b4-8bee-faa029998dbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:38.688020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e512ce-bebd-46b4-8bee-faa029998dbe service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:38.688311 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e512ce-bebd-46b4-8bee-faa029998dbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:38.688592 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e512ce-bebd-46b4-8bee-faa029998dbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:38.696003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b2e512ce-bebd-46b4-8bee-faa029998dbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:38.712302 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e512ce-bebd-46b4-8bee-faa029998dbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:38.714737 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e512ce-bebd-46b4-8bee-faa029998dbe service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:38.724368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1220/2441] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:29:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:29:39.326759 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1b336-8d80-49bf-9032-849fadfc262a None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:39.327281 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1b336-8d80-49bf-9032-849fadfc262a None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:39.327397 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1b336-8d80-49bf-9032-849fadfc262a None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:39.328941 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1222/2442] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jun 23 09:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:29:39.334586 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3d4fd-91ce-424d-a437-418ad8787109 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:39.334947 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3d4fd-91ce-424d-a437-418ad8787109 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:39.335296 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3d4fd-91ce-424d-a437-418ad8787109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:39.373541 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-dce3d4fd-91ce-424d-a437-418ad8787109 None None] MFA Rules not processed for user `ff025cb706d243778468ae4570374690`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:29:39.374781 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-dce3d4fd-91ce-424d-a437-418ad8787109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:39.384090 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1221/2443] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jun 23 09:29:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:29:39.396144 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1b336-8d80-49bf-9032-849fadfc262a None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:39.400936 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d9d1b336-8d80-49bf-9032-849fadfc262a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:39.458384 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1b336-8d80-49bf-9032-849fadfc262a None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:39.462007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8106d1-3c8b-4123-a0eb-2cdaf6f1b433 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36a6a3190dad4948a505bbe4c7095699', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:39.462662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8106d1-3c8b-4123-a0eb-2cdaf6f1b433 service glance] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:39.463197 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8106d1-3c8b-4123-a0eb-2cdaf6f1b433 service glance] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:39.463482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8106d1-3c8b-4123-a0eb-2cdaf6f1b433 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:39.475808 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8106d1-3c8b-4123-a0eb-2cdaf6f1b433 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:39.477612 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8106d1-3c8b-4123-a0eb-2cdaf6f1b433 service glance] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:39.491737 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1223/2444] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:29:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:29:39.829027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b29766-d11b-48b3-aad1-d06cebc4cd07 None None] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:39.829546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b29766-d11b-48b3-aad1-d06cebc4cd07 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:39.829646 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b29766-d11b-48b3-aad1-d06cebc4cd07 None None] PATH_INFO: `/` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:39.831060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1222/2445] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jun 23 09:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:29:39.836030 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc59765-6954-49b1-8451-ba461676ac52 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:39.836325 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc59765-6954-49b1-8451-ba461676ac52 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:39.836615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc59765-6954-49b1-8451-ba461676ac52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:39.860809 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-fdc59765-6954-49b1-8451-ba461676ac52 None None] MFA Rules not processed for user `daed809f511a43ed8d3f88eba10466b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:29:39.862219 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-fdc59765-6954-49b1-8451-ba461676ac52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:39.868936 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1224/2446] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jun 23 09:29:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:29:39.888511 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b29766-d11b-48b3-aad1-d06cebc4cd07 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:39.899949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b29766-d11b-48b3-aad1-d06cebc4cd07 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:39.904106 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704b320-5729-47aa-8657-5035ea82e306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:39.904757 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704b320-5729-47aa-8657-5035ea82e306 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:39.905043 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704b320-5729-47aa-8657-5035ea82e306 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:39.905321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704b320-5729-47aa-8657-5035ea82e306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:39.913070 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704b320-5729-47aa-8657-5035ea82e306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:39.914493 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704b320-5729-47aa-8657-5035ea82e306 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:39.923250 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1223/2447] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:29:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:29:40.008574 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef493b-1280-46ff-9b90-1bdb76a9a016 None None] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:40.008870 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef493b-1280-46ff-9b90-1bdb76a9a016 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:40.009156 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef493b-1280-46ff-9b90-1bdb76a9a016 None None] PATH_INFO: `/` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:40.012703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1225/2448] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jun 23 09:29:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 09:29:40.018363 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafc8f0-b657-4750-b27a-065be2dbd598 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:40.018712 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafc8f0-b657-4750-b27a-065be2dbd598 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:40.018978 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafc8f0-b657-4750-b27a-065be2dbd598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:40.037501 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-5eafc8f0-b657-4750-b27a-065be2dbd598 None None] MFA Rules not processed for user `ff025cb706d243778468ae4570374690`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:29:40.038711 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-5eafc8f0-b657-4750-b27a-065be2dbd598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:40.049220 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1224/2449] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jun 23 09:29:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:29:40.059451 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef493b-1280-46ff-9b90-1bdb76a9a016 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:40.071411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef493b-1280-46ff-9b90-1bdb76a9a016 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:40.075432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1b6b3-538f-4dfe-9225-3d36ab3ebfbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:40.076064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1b6b3-538f-4dfe-9225-3d36ab3ebfbc service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:40.076347 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1b6b3-538f-4dfe-9225-3d36ab3ebfbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:40.076622 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1b6b3-538f-4dfe-9225-3d36ab3ebfbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:40.087686 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1b6b3-538f-4dfe-9225-3d36ab3ebfbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:40.089742 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1b6b3-538f-4dfe-9225-3d36ab3ebfbc service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:40.102187 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1226/2450] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:29:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:29:44.996530 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacd86d-3feb-4d48-848c-31ae78390f73 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:44.996990 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacd86d-3feb-4d48-848c-31ae78390f73 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:44.997114 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacd86d-3feb-4d48-848c-31ae78390f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:45.015785 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-acacd86d-3feb-4d48-848c-31ae78390f73 None None] MFA Rules not processed for user `cfbaba84402d4c31bafcd12133d74c03`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:29:45.016989 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-acacd86d-3feb-4d48-848c-31ae78390f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:45.071539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1225/2451] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jun 23 09:29:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:29:45.093919 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1b6b3-538f-4dfe-9225-3d36ab3ebfbc service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:45.099619 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a3a1b6b3-538f-4dfe-9225-3d36ab3ebfbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:45.165652 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1b6b3-538f-4dfe-9225-3d36ab3ebfbc service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:45.173501 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce03cd-606e-4f0f-9eb3-63dd577ea1d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b1331b4f0924bb8b137a22a33c9d130', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:45.173501 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce03cd-606e-4f0f-9eb3-63dd577ea1d9 service swift] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:45.173501 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce03cd-606e-4f0f-9eb3-63dd577ea1d9 service swift] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:45.173501 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce03cd-606e-4f0f-9eb3-63dd577ea1d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:45.180361 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ce03cd-606e-4f0f-9eb3-63dd577ea1d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:45.182514 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ce03cd-606e-4f0f-9eb3-63dd577ea1d9 service swift] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:45.190738 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1227/2452] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jun 23 09:29:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 09:29:49.441276 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacd86d-3feb-4d48-848c-31ae78390f73 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:29:49.471309 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacd86d-3feb-4d48-848c-31ae78390f73 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:29:49.480097 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509ef08-7a7b-44da-8e0d-426c4c92982b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:29:49.480776 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509ef08-7a7b-44da-8e0d-426c4c92982b service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:29:49.481064 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509ef08-7a7b-44da-8e0d-426c4c92982b service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:29:49.481383 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509ef08-7a7b-44da-8e0d-426c4c92982b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:29:49.483360 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-0509ef08-7a7b-44da-8e0d-426c4c92982b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:29:49.497239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0509ef08-7a7b-44da-8e0d-426c4c92982b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:29:49.504972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0509ef08-7a7b-44da-8e0d-426c4c92982b service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:29:49.521144 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1226/2453] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:29:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:29:55.187463 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=87 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:29:59.516058 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=80 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:30:01.711309 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aded67e-b881-4442-80df-4b71a9e9af83 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:30:01.711859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aded67e-b881-4442-80df-4b71a9e9af83 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:30:01.711962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aded67e-b881-4442-80df-4b71a9e9af83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:30:01.733347 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-2aded67e-b881-4442-80df-4b71a9e9af83 None None] MFA Rules not processed for user `cfbaba84402d4c31bafcd12133d74c03`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:30:01.734673 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-2aded67e-b881-4442-80df-4b71a9e9af83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:30:01.742674 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1228/2454] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jun 23 09:30:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:30:01.755926 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509ef08-7a7b-44da-8e0d-426c4c92982b service nova] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:30:01.769839 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509ef08-7a7b-44da-8e0d-426c4c92982b service nova] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:30:01.775412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613e775-6cb0-4317-9010-0e9d70ce56dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b1331b4f0924bb8b137a22a33c9d130', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:30:01.776187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613e775-6cb0-4317-9010-0e9d70ce56dc service swift] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:30:01.776483 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613e775-6cb0-4317-9010-0e9d70ce56dc service swift] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:30:01.776789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613e775-6cb0-4317-9010-0e9d70ce56dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:30:01.786403 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6613e775-6cb0-4317-9010-0e9d70ce56dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:30:01.788056 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6613e775-6cb0-4317-9010-0e9d70ce56dc service swift] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:30:01.798699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1227/2455] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jun 23 09:30:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 09:30:04.228246 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0c3a3-2c02-41ef-b20e-7358e7095927 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:30:04.228743 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0c3a3-2c02-41ef-b20e-7358e7095927 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:30:04.228873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0c3a3-2c02-41ef-b20e-7358e7095927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:30:04.246629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-c1d0c3a3-2c02-41ef-b20e-7358e7095927 None None] MFA Rules not processed for user `ad5452341ccc40bc94b38369dae4f2b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:30:04.247853 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-c1d0c3a3-2c02-41ef-b20e-7358e7095927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:30:04.302731 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1229/2456] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:30:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:30:11.794974 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=6 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:30:14.302719 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=28 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:30:51.699031 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613e775-6cb0-4317-9010-0e9d70ce56dc service swift] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:30:51.708297 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613e775-6cb0-4317-9010-0e9d70ce56dc service swift] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:30:51.712476 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2fd167-0843-47f5-9510-229f115397ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:30:51.713124 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fd167-0843-47f5-9510-229f115397ef service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:30:51.713415 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fd167-0843-47f5-9510-229f115397ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:30:51.713690 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fd167-0843-47f5-9510-229f115397ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:30:51.715467 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-0a2fd167-0843-47f5-9510-229f115397ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:30:51.727126 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fd167-0843-47f5-9510-229f115397ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:30:51.728534 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fd167-0843-47f5-9510-229f115397ef service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:30:51.738258 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1228/2457] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:30:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:30:55.195630 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0c3a3-2c02-41ef-b20e-7358e7095927 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:30:55.209370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0c3a3-2c02-41ef-b20e-7358e7095927 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:30:55.214895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ef80e-6c28-46fb-9f5a-adfe80ffddad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63b20ceca40e6bceb56eda8ae9686', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:30:55.215604 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ef80e-6c28-46fb-9f5a-adfe80ffddad service placement] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:30:55.215892 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ef80e-6c28-46fb-9f5a-adfe80ffddad service placement] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:30:55.216169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ef80e-6c28-46fb-9f5a-adfe80ffddad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:30:55.218006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5e7ef80e-6c28-46fb-9f5a-adfe80ffddad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:30:55.230624 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ef80e-6c28-46fb-9f5a-adfe80ffddad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:30:55.232331 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ef80e-6c28-46fb-9f5a-adfe80ffddad service placement] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:30:55.243148 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1230/2458] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:30:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:30:55.346740 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2fd167-0843-47f5-9510-229f115397ef service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:30:55.348402 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-0a2fd167-0843-47f5-9510-229f115397ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:30:55.361877 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2fd167-0843-47f5-9510-229f115397ef service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:30:55.366732 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659ef8f-6408-463b-9571-651e3fe4b547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:30:55.367668 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659ef8f-6408-463b-9571-651e3fe4b547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:30:55.367960 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659ef8f-6408-463b-9571-651e3fe4b547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:30:55.368808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659ef8f-6408-463b-9571-651e3fe4b547 admin admin] PATH_INFO: `/v3/users/30bc3dba44ed4d14a67e56aa640bf9ca` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:30:55.369988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659ef8f-6408-463b-9571-651e3fe4b547 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30bc3dba44ed4d14a67e56aa640bf9ca)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:30:55.371971 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659ef8f-6408-463b-9571-651e3fe4b547 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:30:55.576702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-6659ef8f-6408-463b-9571-651e3fe4b547 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30bc3dba44ed4d14a67e56aa640bf9ca'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:30:55.583645 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-6659ef8f-6408-463b-9571-651e3fe4b547 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30bc3dba44ed4d14a67e56aa640bf9ca'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:30:55.584486 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-6659ef8f-6408-463b-9571-651e3fe4b547 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30bc3dba44ed4d14a67e56aa640bf9ca'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:30:55.588613 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-6659ef8f-6408-463b-9571-651e3fe4b547 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30bc3dba44ed4d14a67e56aa640bf9ca'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:30:55.597788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-6659ef8f-6408-463b-9571-651e3fe4b547 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30bc3dba44ed4d14a67e56aa640bf9ca'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:30:55.708485 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1229/2459] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:30:55 2022] DELETE /identity/v3/users/30bc3dba44ed4d14a67e56aa640bf9ca => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:30:55.970376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ef80e-6c28-46fb-9f5a-adfe80ffddad service placement] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:30:55.986622 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ef80e-6c28-46fb-9f5a-adfe80ffddad service placement] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:30:56.052125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9112893-402f-4920-bdea-7077ac0ef2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:30:56.052978 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9112893-402f-4920-bdea-7077ac0ef2b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:30:56.053263 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9112893-402f-4920-bdea-7077ac0ef2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:30:56.053540 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9112893-402f-4920-bdea-7077ac0ef2b9 admin admin] PATH_INFO: `/v3/projects/29ece34041e04c78868e53d90b658a82` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:30:56.054707 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9112893-402f-4920-bdea-7077ac0ef2b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29ece34041e04c78868e53d90b658a82)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:30:56.056686 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9112893-402f-4920-bdea-7077ac0ef2b9 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:30:56.164668 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-c9112893-402f-4920-bdea-7077ac0ef2b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29ece34041e04c78868e53d90b658a82', 'request_id': 'req-c9112893-402f-4920-bdea-7077ac0ef2b9'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:30:56.171143 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-c9112893-402f-4920-bdea-7077ac0ef2b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29ece34041e04c78868e53d90b658a82', 'request_id': 'req-c9112893-402f-4920-bdea-7077ac0ef2b9'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:30:56.242748 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-c9112893-402f-4920-bdea-7077ac0ef2b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29ece34041e04c78868e53d90b658a82', 'request_id': 'req-c9112893-402f-4920-bdea-7077ac0ef2b9'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:30:56.254581 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-c9112893-402f-4920-bdea-7077ac0ef2b9 admin admin] The token cache is being invalidate because project 29ece34041e04c78868e53d90b658a82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:30:56.254870 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-c9112893-402f-4920-bdea-7077ac0ef2b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:30:56.255565 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-c9112893-402f-4920-bdea-7077ac0ef2b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:30:56.257133 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1231/2460] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:30:55 2022] DELETE /identity/v3/projects/29ece34041e04c78868e53d90b658a82 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:30:56.262671 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8072d-a431-4b45-bd3a-6b657ba49f9d None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:30:56.262998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8072d-a431-4b45-bd3a-6b657ba49f9d None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:30:56.263285 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8072d-a431-4b45-bd3a-6b657ba49f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:30:56.282236 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-eeb8072d-a431-4b45-bd3a-6b657ba49f9d None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:30:56.283611 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-eeb8072d-a431-4b45-bd3a-6b657ba49f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:30:56.367544 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1230/2461] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:30:56.376285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c39a03-dafe-4b86-a9fe-7660b35567d3 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:30:56.377567 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c39a03-dafe-4b86-a9fe-7660b35567d3 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:30:56.377567 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c39a03-dafe-4b86-a9fe-7660b35567d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:30:56.401380 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-06c39a03-dafe-4b86-a9fe-7660b35567d3 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:30:56.402607 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-06c39a03-dafe-4b86-a9fe-7660b35567d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:30:56.410823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1232/2462] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:30:56.415343 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8072d-a431-4b45-bd3a-6b657ba49f9d None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:30:56.425254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8072d-a431-4b45-bd3a-6b657ba49f9d None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:30:56.430268 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36c712-25e8-463a-8747-fdd9817c6249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:30:56.436125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36c712-25e8-463a-8747-fdd9817c6249 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:30:56.436125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36c712-25e8-463a-8747-fdd9817c6249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:30:56.436125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36c712-25e8-463a-8747-fdd9817c6249 admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:30:56.436125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36c712-25e8-463a-8747-fdd9817c6249 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:30:56.436125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36c712-25e8-463a-8747-fdd9817c6249 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:30:56.436125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36c712-25e8-463a-8747-fdd9817c6249 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:30:56.448242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1231/2463] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:30:56.452792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c39a03-dafe-4b86-a9fe-7660b35567d3 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:30:56.462286 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c39a03-dafe-4b86-a9fe-7660b35567d3 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:30:56.472807 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605fb5f-eee7-49f7-9ebe-73c3250946ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:30:56.475131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605fb5f-eee7-49f7-9ebe-73c3250946ea admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:30:56.475131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605fb5f-eee7-49f7-9ebe-73c3250946ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:30:56.475131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605fb5f-eee7-49f7-9ebe-73c3250946ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:30:56.475131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9605fb5f-eee7-49f7-9ebe-73c3250946ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:30:56.476705 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9605fb5f-eee7-49f7-9ebe-73c3250946ea admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:30:56.593030 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1233/2464] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:30:56 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:30:56.597257 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36c712-25e8-463a-8747-fdd9817c6249 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:30:56.609665 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36c712-25e8-463a-8747-fdd9817c6249 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:30:56.660272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:30:56.716218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545aae9-3f17-4dac-bd46-e9567e08410c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:30:56.716218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545aae9-3f17-4dac-bd46-e9567e08410c admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:30:56.716218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545aae9-3f17-4dac-bd46-e9567e08410c admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:30:56.716218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545aae9-3f17-4dac-bd46-e9567e08410c admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:30:56.718189 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545aae9-3f17-4dac-bd46-e9567e08410c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:30:56.718189 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545aae9-3f17-4dac-bd46-e9567e08410c admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:30:56.821091 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1232/2465] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:30:56 2022] POST /identity/v3/users => generated 292 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:30:56.827050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605fb5f-eee7-49f7-9ebe-73c3250946ea admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:30:56.843190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605fb5f-eee7-49f7-9ebe-73c3250946ea admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:30:56.860016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092d7eb-46c2-44af-8aed-90b0ae22b5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:30:56.861082 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092d7eb-46c2-44af-8aed-90b0ae22b5d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:30:56.861536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092d7eb-46c2-44af-8aed-90b0ae22b5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:30:56.861834 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092d7eb-46c2-44af-8aed-90b0ae22b5d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:30:56.863162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6092d7eb-46c2-44af-8aed-90b0ae22b5d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:30:56.864807 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6092d7eb-46c2-44af-8aed-90b0ae22b5d0 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:30:56.878970 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1234/2466] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:30:56.880594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545aae9-3f17-4dac-bd46-e9567e08410c admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:30:56.894121 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545aae9-3f17-4dac-bd46-e9567e08410c admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:30:56.904918 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb3eac-12d3-4e07-9440-9727a6ac5e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:30:56.906822 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb3eac-12d3-4e07-9440-9727a6ac5e4c admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:30:56.907396 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb3eac-12d3-4e07-9440-9727a6ac5e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:30:56.907848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb3eac-12d3-4e07-9440-9727a6ac5e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:30:56.908905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb3eac-12d3-4e07-9440-9727a6ac5e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:30:56.910998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb3eac-12d3-4e07-9440-9727a6ac5e4c admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:30:56.947299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1233/2467] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:30:56.953096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092d7eb-46c2-44af-8aed-90b0ae22b5d0 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:30:56.964117 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092d7eb-46c2-44af-8aed-90b0ae22b5d0 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:30:56.969339 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e8666-d137-4cca-a3c7-dcaa71c096f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:30:56.970453 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e8666-d137-4cca-a3c7-dcaa71c096f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:30:56.970870 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e8666-d137-4cca-a3c7-dcaa71c096f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:30:56.973412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e8666-d137-4cca-a3c7-dcaa71c096f0 admin admin] PATH_INFO: `/v3/projects/dd36fda2fe544388b2392365bb4cb04d/users/a73ba0b50a5244208b64d73268a516e9/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:30:56.988161 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e8666-d137-4cca-a3c7-dcaa71c096f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd36fda2fe544388b2392365bb4cb04d, user_id=a73ba0b50a5244208b64d73268a516e9, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:30:56.991395 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e8666-d137-4cca-a3c7-dcaa71c096f0 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:30:57.072782 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1235/2468] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:30:56 2022] PUT /identity/v3/projects/dd36fda2fe544388b2392365bb4cb04d/users/a73ba0b50a5244208b64d73268a516e9/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:30:57.084432 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f18a6f-65d8-4468-abf6-76c41842e1d8 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:30:57.084729 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f18a6f-65d8-4468-abf6-76c41842e1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:30:57.085033 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f18a6f-65d8-4468-abf6-76c41842e1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:30:57.120844 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-11f18a6f-65d8-4468-abf6-76c41842e1d8 None None] MFA Rules not processed for user `a73ba0b50a5244208b64d73268a516e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:30:57.127776 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-11f18a6f-65d8-4468-abf6-76c41842e1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:30:57.223716 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1234/2469] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:30:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:30:57.235003 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e8666-d137-4cca-a3c7-dcaa71c096f0 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:30:57.239620 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-281e8666-d137-4cca-a3c7-dcaa71c096f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:30:57.259709 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:30:57.351352 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e8666-d137-4cca-a3c7-dcaa71c096f0 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:30:57.359810 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6dada-943a-4393-a08e-51e4a424ffb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:30:57.360497 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6dada-943a-4393-a08e-51e4a424ffb3 service nova] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:30:57.360799 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6dada-943a-4393-a08e-51e4a424ffb3 service nova] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:30:57.361093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6dada-943a-4393-a08e-51e4a424ffb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:30:57.371846 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6dada-943a-4393-a08e-51e4a424ffb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:30:57.373923 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6dada-943a-4393-a08e-51e4a424ffb3 service nova] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:30:57.385258 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1236/2470] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:30:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:31:02.408975 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f18a6f-65d8-4468-abf6-76c41842e1d8 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:02.410392 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-11f18a6f-65d8-4468-abf6-76c41842e1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:02.426341 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f18a6f-65d8-4468-abf6-76c41842e1d8 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:02.432250 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46adc74b-9719-4c77-b895-b017d03555b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:02.432888 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adc74b-9719-4c77-b895-b017d03555b6 service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:02.433176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adc74b-9719-4c77-b895-b017d03555b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:02.433456 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adc74b-9719-4c77-b895-b017d03555b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:02.435208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-46adc74b-9719-4c77-b895-b017d03555b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:02.496071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46adc74b-9719-4c77-b895-b017d03555b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:02.497692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46adc74b-9719-4c77-b895-b017d03555b6 service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:02.506792 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1235/2471] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:31:06.422079 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6dada-943a-4393-a08e-51e4a424ffb3 service nova] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:06.481600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6dada-943a-4393-a08e-51e4a424ffb3 service nova] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:06.492715 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5fbff-3d83-4307-b240-fb74d3747c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:06.494528 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5fbff-3d83-4307-b240-fb74d3747c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:06.494822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5fbff-3d83-4307-b240-fb74d3747c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:06.495362 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5fbff-3d83-4307-b240-fb74d3747c16 admin admin] PATH_INFO: `/v3/users/a73ba0b50a5244208b64d73268a516e9` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:06.501450 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5fbff-3d83-4307-b240-fb74d3747c16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a73ba0b50a5244208b64d73268a516e9)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:06.501450 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5fbff-3d83-4307-b240-fb74d3747c16 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:06.656224 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-10a5fbff-3d83-4307-b240-fb74d3747c16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a73ba0b50a5244208b64d73268a516e9'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:06.663308 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-10a5fbff-3d83-4307-b240-fb74d3747c16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a73ba0b50a5244208b64d73268a516e9'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:06.663974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-10a5fbff-3d83-4307-b240-fb74d3747c16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a73ba0b50a5244208b64d73268a516e9'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:06.664616 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-10a5fbff-3d83-4307-b240-fb74d3747c16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a73ba0b50a5244208b64d73268a516e9'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:06.676343 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-10a5fbff-3d83-4307-b240-fb74d3747c16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a73ba0b50a5244208b64d73268a516e9'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:06.774038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1237/2472] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:31:06 2022] DELETE /identity/v3/users/a73ba0b50a5244208b64d73268a516e9 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:06.778619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756112c-fa42-4c0d-af3b-f09dc7543df6 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:06.779135 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756112c-fa42-4c0d-af3b-f09dc7543df6 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:06.779237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756112c-fa42-4c0d-af3b-f09dc7543df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:06.796694 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-7756112c-fa42-4c0d-af3b-f09dc7543df6 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:06.797956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7756112c-fa42-4c0d-af3b-f09dc7543df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:06.846079 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:06.858078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1236/2473] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:06.868471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5fbff-3d83-4307-b240-fb74d3747c16 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:06.870091 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-10a5fbff-3d83-4307-b240-fb74d3747c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:06.880949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:06.925835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5fbff-3d83-4307-b240-fb74d3747c16 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:06.929480 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc50a4-88ee-46f8-b325-f2c6aaa98d0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:06.930125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc50a4-88ee-46f8-b325-f2c6aaa98d0a service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:06.930903 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc50a4-88ee-46f8-b325-f2c6aaa98d0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:06.931220 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc50a4-88ee-46f8-b325-f2c6aaa98d0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:06.940579 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc50a4-88ee-46f8-b325-f2c6aaa98d0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:06.942038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc50a4-88ee-46f8-b325-f2c6aaa98d0a service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:06.953552 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1238/2474] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:31:07.419235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756112c-fa42-4c0d-af3b-f09dc7543df6 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:07.420685 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7756112c-fa42-4c0d-af3b-f09dc7543df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:07.434559 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756112c-fa42-4c0d-af3b-f09dc7543df6 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:07.440648 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f83e07-ab29-420c-a789-f9c7da5e7ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:07.441538 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f83e07-ab29-420c-a789-f9c7da5e7ce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:07.441833 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f83e07-ab29-420c-a789-f9c7da5e7ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:07.442147 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f83e07-ab29-420c-a789-f9c7da5e7ce9 admin admin] PATH_INFO: `/v3/projects/dd36fda2fe544388b2392365bb4cb04d` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:07.443554 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f83e07-ab29-420c-a789-f9c7da5e7ce9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd36fda2fe544388b2392365bb4cb04d)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:07.445503 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f83e07-ab29-420c-a789-f9c7da5e7ce9 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:07.587172 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-93f83e07-ab29-420c-a789-f9c7da5e7ce9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd36fda2fe544388b2392365bb4cb04d', 'request_id': 'req-93f83e07-ab29-420c-a789-f9c7da5e7ce9'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:07.593688 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-93f83e07-ab29-420c-a789-f9c7da5e7ce9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd36fda2fe544388b2392365bb4cb04d', 'request_id': 'req-93f83e07-ab29-420c-a789-f9c7da5e7ce9'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:07.600237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-93f83e07-ab29-420c-a789-f9c7da5e7ce9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd36fda2fe544388b2392365bb4cb04d', 'request_id': 'req-93f83e07-ab29-420c-a789-f9c7da5e7ce9'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:07.689037 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-93f83e07-ab29-420c-a789-f9c7da5e7ce9 admin admin] The token cache is being invalidate because project dd36fda2fe544388b2392365bb4cb04d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:31:07.689372 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-93f83e07-ab29-420c-a789-f9c7da5e7ce9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:07.699011 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-93f83e07-ab29-420c-a789-f9c7da5e7ce9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:07.699011 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1237/2475] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:31:07 2022] DELETE /identity/v3/projects/dd36fda2fe544388b2392365bb4cb04d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:07.699206 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec756a-2828-4dd3-9adf-1815423dab7b None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:07.699206 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec756a-2828-4dd3-9adf-1815423dab7b None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:07.699206 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec756a-2828-4dd3-9adf-1815423dab7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:07.715299 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-68ec756a-2828-4dd3-9adf-1815423dab7b None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:07.716764 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-68ec756a-2828-4dd3-9adf-1815423dab7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:07.783347 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1239/2476] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:31:07.791744 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942a564-5708-4122-bbe4-9ad9b44616f4 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:07.792280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942a564-5708-4122-bbe4-9ad9b44616f4 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:07.792280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942a564-5708-4122-bbe4-9ad9b44616f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:07.810439 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-9942a564-5708-4122-bbe4-9ad9b44616f4 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:07.811689 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-9942a564-5708-4122-bbe4-9ad9b44616f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:07.819527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1238/2477] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:31:07.823886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec756a-2828-4dd3-9adf-1815423dab7b None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:07.833684 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec756a-2828-4dd3-9adf-1815423dab7b None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:07.838841 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa45bd1-f5ca-42a9-9207-9acd922fe6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:07.839818 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa45bd1-f5ca-42a9-9207-9acd922fe6c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:07.840250 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa45bd1-f5ca-42a9-9207-9acd922fe6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:07.840568 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa45bd1-f5ca-42a9-9207-9acd922fe6c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:07.841276 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa45bd1-f5ca-42a9-9207-9acd922fe6c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:31:07.841599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa45bd1-f5ca-42a9-9207-9acd922fe6c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:07.843240 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa45bd1-f5ca-42a9-9207-9acd922fe6c1 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:07.858970 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1240/2478] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:31:07.863709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942a564-5708-4122-bbe4-9ad9b44616f4 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:07.873733 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942a564-5708-4122-bbe4-9ad9b44616f4 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:07.878804 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777b578-196f-4399-a1ff-60810c205677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:07.879545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777b578-196f-4399-a1ff-60810c205677 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:07.879835 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777b578-196f-4399-a1ff-60810c205677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:07.880118 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777b578-196f-4399-a1ff-60810c205677 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:07.880886 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777b578-196f-4399-a1ff-60810c205677 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:07.882824 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777b578-196f-4399-a1ff-60810c205677 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:07.948532 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1239/2479] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:31:07 2022] POST /identity/v3/projects => generated 357 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:07.952640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa45bd1-f5ca-42a9-9207-9acd922fe6c1 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:07.963700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa45bd1-f5ca-42a9-9207-9acd922fe6c1 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:08.027119 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011453c-eead-498e-88fe-ceb8de8ef95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:08.027802 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011453c-eead-498e-88fe-ceb8de8ef95d admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:08.028083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011453c-eead-498e-88fe-ceb8de8ef95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:08.028358 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011453c-eead-498e-88fe-ceb8de8ef95d admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:08.029061 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011453c-eead-498e-88fe-ceb8de8ef95d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:08.031043 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011453c-eead-498e-88fe-ceb8de8ef95d admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:08.089071 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1241/2480] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:31:07 2022] POST /identity/v3/users => generated 279 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:31:08.093148 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777b578-196f-4399-a1ff-60810c205677 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:08.104040 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777b578-196f-4399-a1ff-60810c205677 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:08.108642 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9da3af-625e-4763-b46d-6e18fdb4a4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:08.109299 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9da3af-625e-4763-b46d-6e18fdb4a4f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:08.109584 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9da3af-625e-4763-b46d-6e18fdb4a4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:08.109863 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9da3af-625e-4763-b46d-6e18fdb4a4f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:08.110590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9da3af-625e-4763-b46d-6e18fdb4a4f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:08.112480 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9da3af-625e-4763-b46d-6e18fdb4a4f8 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:08.122788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1240/2481] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:08.129174 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011453c-eead-498e-88fe-ceb8de8ef95d admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:08.139550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011453c-eead-498e-88fe-ceb8de8ef95d admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:08.144118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1071d9-5808-4949-8cc1-2552a63fde58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:08.144770 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1071d9-5808-4949-8cc1-2552a63fde58 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:08.145062 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1071d9-5808-4949-8cc1-2552a63fde58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:08.145343 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1071d9-5808-4949-8cc1-2552a63fde58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:08.146072 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1071d9-5808-4949-8cc1-2552a63fde58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:08.147711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1071d9-5808-4949-8cc1-2552a63fde58 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:08.158621 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1242/2482] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:08.162716 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9da3af-625e-4763-b46d-6e18fdb4a4f8 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:08.172929 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9da3af-625e-4763-b46d-6e18fdb4a4f8 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:08.177677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a649ea1-d93c-42e1-99c0-e1cc7757e7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:08.178556 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a649ea1-d93c-42e1-99c0-e1cc7757e7af admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:08.178842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a649ea1-d93c-42e1-99c0-e1cc7757e7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:08.179227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a649ea1-d93c-42e1-99c0-e1cc7757e7af admin admin] PATH_INFO: `/v3/projects/e7fc8503ca8a423c9776eb92c9c79d53/users/38ba424297a34ddc956208e6adeca85c/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:08.196902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a649ea1-d93c-42e1-99c0-e1cc7757e7af admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7fc8503ca8a423c9776eb92c9c79d53, user_id=38ba424297a34ddc956208e6adeca85c, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:08.199669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a649ea1-d93c-42e1-99c0-e1cc7757e7af admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:08.266012 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1241/2483] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:08 2022] PUT /identity/v3/projects/e7fc8503ca8a423c9776eb92c9c79d53/users/38ba424297a34ddc956208e6adeca85c/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:08.277510 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af4eb5-0a90-48b9-a3be-13d81b6c5753 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:08.277799 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af4eb5-0a90-48b9-a3be-13d81b6c5753 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:08.278077 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af4eb5-0a90-48b9-a3be-13d81b6c5753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:08.303897 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-01af4eb5-0a90-48b9-a3be-13d81b6c5753 None None] MFA Rules not processed for user `38ba424297a34ddc956208e6adeca85c`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:08.305097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-01af4eb5-0a90-48b9-a3be-13d81b6c5753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:08.310638 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:08.370421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1243/2484] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:08 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:08.374955 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a649ea1-d93c-42e1-99c0-e1cc7757e7af admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:08.437397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a649ea1-d93c-42e1-99c0-e1cc7757e7af admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:08.442339 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d36384-93d3-4b1e-9c0d-45f37cc71f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:08.443039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d36384-93d3-4b1e-9c0d-45f37cc71f4b admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:08.443328 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d36384-93d3-4b1e-9c0d-45f37cc71f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:08.443605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d36384-93d3-4b1e-9c0d-45f37cc71f4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:08.444304 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d36384-93d3-4b1e-9c0d-45f37cc71f4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:08.447255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d36384-93d3-4b1e-9c0d-45f37cc71f4b admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:08.587906 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:08.597463 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1242/2485] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:31:08 2022] POST /identity/v3/projects => generated 357 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:08.601027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af4eb5-0a90-48b9-a3be-13d81b6c5753 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:08.613066 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af4eb5-0a90-48b9-a3be-13d81b6c5753 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:08.675183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b717dc-d733-4262-8c74-835726af50a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:08.675896 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b717dc-d733-4262-8c74-835726af50a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:08.676186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b717dc-d733-4262-8c74-835726af50a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:08.676498 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b717dc-d733-4262-8c74-835726af50a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:08.677200 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b717dc-d733-4262-8c74-835726af50a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:08.679251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b717dc-d733-4262-8c74-835726af50a6 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:08.865662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1244/2486] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:31:08 2022] POST /identity/v3/users => generated 285 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:31:08.870158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d36384-93d3-4b1e-9c0d-45f37cc71f4b admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:08.880504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d36384-93d3-4b1e-9c0d-45f37cc71f4b admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:08.885204 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b67b13-7997-453d-b7f7-2de5693e2509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:08.885870 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b67b13-7997-453d-b7f7-2de5693e2509 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:08.886159 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b67b13-7997-453d-b7f7-2de5693e2509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:08.886439 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b67b13-7997-453d-b7f7-2de5693e2509 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:08.887229 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b67b13-7997-453d-b7f7-2de5693e2509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:08.888828 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b67b13-7997-453d-b7f7-2de5693e2509 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:08.903235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1243/2487] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:08.906359 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b717dc-d733-4262-8c74-835726af50a6 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:08.914179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b717dc-d733-4262-8c74-835726af50a6 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:08.919347 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b709f-25a0-48ed-984e-d7197c21cce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:08.920259 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b709f-25a0-48ed-984e-d7197c21cce1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:08.920595 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b709f-25a0-48ed-984e-d7197c21cce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:08.920896 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b709f-25a0-48ed-984e-d7197c21cce1 admin admin] PATH_INFO: `/v3/projects/d957cdcf5fb34d6c951449d892cc00de/users/14185027c8884ee28a1ded4464e418d6/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:08.935968 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b709f-25a0-48ed-984e-d7197c21cce1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d957cdcf5fb34d6c951449d892cc00de, user_id=14185027c8884ee28a1ded4464e418d6, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:08.938628 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b709f-25a0-48ed-984e-d7197c21cce1 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:09.048081 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1245/2488] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:08 2022] PUT /identity/v3/projects/d957cdcf5fb34d6c951449d892cc00de/users/14185027c8884ee28a1ded4464e418d6/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:31:09.052122 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b67b13-7997-453d-b7f7-2de5693e2509 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:09.062008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b67b13-7997-453d-b7f7-2de5693e2509 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:09.117669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f88002-1620-4efd-a995-a0f26df69ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:09.118267 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f88002-1620-4efd-a995-a0f26df69ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:09.118561 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f88002-1620-4efd-a995-a0f26df69ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:09.118836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f88002-1620-4efd-a995-a0f26df69ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:09.119593 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f88002-1620-4efd-a995-a0f26df69ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:09.121198 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f88002-1620-4efd-a995-a0f26df69ca6 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:09.130838 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1244/2489] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:09.135259 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b709f-25a0-48ed-984e-d7197c21cce1 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:09.145803 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b709f-25a0-48ed-984e-d7197c21cce1 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:09.151020 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35703002-4852-42b3-96f4-c24c65803757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:09.151939 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35703002-4852-42b3-96f4-c24c65803757 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:09.152257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35703002-4852-42b3-96f4-c24c65803757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:09.152885 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35703002-4852-42b3-96f4-c24c65803757 admin admin] PATH_INFO: `/v3/projects/d957cdcf5fb34d6c951449d892cc00de/users/14185027c8884ee28a1ded4464e418d6/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:09.154903 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35703002-4852-42b3-96f4-c24c65803757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d957cdcf5fb34d6c951449d892cc00de, user_id=14185027c8884ee28a1ded4464e418d6, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:09.157669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35703002-4852-42b3-96f4-c24c65803757 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:09.169340 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1246/2490] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:09 2022] PUT /identity/v3/projects/d957cdcf5fb34d6c951449d892cc00de/users/14185027c8884ee28a1ded4464e418d6/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:09.179546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb549e3-d686-4aeb-a6ad-8f379fc0be14 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:09.179848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb549e3-d686-4aeb-a6ad-8f379fc0be14 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:09.180735 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb549e3-d686-4aeb-a6ad-8f379fc0be14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:09.207237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-9cb549e3-d686-4aeb-a6ad-8f379fc0be14 None None] MFA Rules not processed for user `14185027c8884ee28a1ded4464e418d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:09.208474 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-9cb549e3-d686-4aeb-a6ad-8f379fc0be14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:09.278576 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1245/2491] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:09 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:09.289801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35703002-4852-42b3-96f4-c24c65803757 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:09.292129 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-35703002-4852-42b3-96f4-c24c65803757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:09.347301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35703002-4852-42b3-96f4-c24c65803757 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:09.350872 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c985cf9-a260-454f-9c15-61d1e071b280 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:09.351538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c985cf9-a260-454f-9c15-61d1e071b280 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:09.351823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c985cf9-a260-454f-9c15-61d1e071b280 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:09.352097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c985cf9-a260-454f-9c15-61d1e071b280 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:09.362498 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c985cf9-a260-454f-9c15-61d1e071b280 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:09.363985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c985cf9-a260-454f-9c15-61d1e071b280 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:09.373728 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1247/2492] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:09 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:31:09.633232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb549e3-d686-4aeb-a6ad-8f379fc0be14 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:09.646942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb549e3-d686-4aeb-a6ad-8f379fc0be14 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:09.650396 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a811a0-f985-416c-9071-74aae9589b09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:09.651331 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a811a0-f985-416c-9071-74aae9589b09 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:09.651621 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a811a0-f985-416c-9071-74aae9589b09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:09.651904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a811a0-f985-416c-9071-74aae9589b09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:09.724550 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a811a0-f985-416c-9071-74aae9589b09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:09.726005 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a811a0-f985-416c-9071-74aae9589b09 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:09.735524 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1246/2493] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:09 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:31:09.831693 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c37da-2913-466f-b32a-8a87eea17fd3 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:09.832974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c37da-2913-466f-b32a-8a87eea17fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:09.832974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c37da-2913-466f-b32a-8a87eea17fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:09.838229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:09.850633 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-1d5c37da-2913-466f-b32a-8a87eea17fd3 None None] MFA Rules not processed for user `cfbaba84402d4c31bafcd12133d74c03`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:09.855917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-1d5c37da-2913-466f-b32a-8a87eea17fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:09.906576 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1248/2494] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jun 23 09:31:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:10.041237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a811a0-f985-416c-9071-74aae9589b09 service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:10.042630 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-c4a811a0-f985-416c-9071-74aae9589b09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:10.052764 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:10.117698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a811a0-f985-416c-9071-74aae9589b09 service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:10.124814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a957e-ad89-4cae-9183-4ded906b63c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b1331b4f0924bb8b137a22a33c9d130', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:10.125578 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a957e-ad89-4cae-9183-4ded906b63c9 service swift] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:10.125869 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a957e-ad89-4cae-9183-4ded906b63c9 service swift] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:10.126155 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a957e-ad89-4cae-9183-4ded906b63c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:10.135408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a957e-ad89-4cae-9183-4ded906b63c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:10.137086 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a957e-ad89-4cae-9183-4ded906b63c9 service swift] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:10.146447 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1247/2495] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jun 23 09:31:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 09:31:15.511046 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c37da-2913-466f-b32a-8a87eea17fd3 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:15.513378 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-1d5c37da-2913-466f-b32a-8a87eea17fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:15.584811 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c37da-2913-466f-b32a-8a87eea17fd3 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:15.589837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f737fb-0780-40b4-becd-2fe0854e758e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63b20ceca40e6bceb56eda8ae9686', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:15.590490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f737fb-0780-40b4-becd-2fe0854e758e service placement] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:15.590858 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f737fb-0780-40b4-becd-2fe0854e758e service placement] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:15.591180 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f737fb-0780-40b4-becd-2fe0854e758e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:15.592942 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-c1f737fb-0780-40b4-becd-2fe0854e758e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:15.613820 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f737fb-0780-40b4-becd-2fe0854e758e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:15.613820 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f737fb-0780-40b4-becd-2fe0854e758e service placement] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:15.618889 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1249/2496] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:31:19.639397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a957e-ad89-4cae-9183-4ded906b63c9 service swift] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:19.641052 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-539a957e-ad89-4cae-9183-4ded906b63c9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:19.712096 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a957e-ad89-4cae-9183-4ded906b63c9 service swift] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:19.720012 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b4e33-c811-435e-8169-58ceec76c972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:19.721045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b4e33-c811-435e-8169-58ceec76c972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:19.721433 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b4e33-c811-435e-8169-58ceec76c972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:19.721836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b4e33-c811-435e-8169-58ceec76c972 admin admin] PATH_INFO: `/v3/users/fc1a2967d343413d8eb72de6a49ba653` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:19.723235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b4e33-c811-435e-8169-58ceec76c972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc1a2967d343413d8eb72de6a49ba653)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:19.725512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b4e33-c811-435e-8169-58ceec76c972 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:19.891492 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-896b4e33-c811-435e-8169-58ceec76c972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc1a2967d343413d8eb72de6a49ba653'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:19.898457 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-896b4e33-c811-435e-8169-58ceec76c972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc1a2967d343413d8eb72de6a49ba653'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:19.899866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-896b4e33-c811-435e-8169-58ceec76c972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc1a2967d343413d8eb72de6a49ba653'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:19.900490 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-896b4e33-c811-435e-8169-58ceec76c972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc1a2967d343413d8eb72de6a49ba653'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:19.912895 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-896b4e33-c811-435e-8169-58ceec76c972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc1a2967d343413d8eb72de6a49ba653'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:19.962989 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1248/2497] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:31:19 2022] DELETE /identity/v3/users/fc1a2967d343413d8eb72de6a49ba653 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:20.233105 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f737fb-0780-40b4-becd-2fe0854e758e service placement] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:20.245445 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f737fb-0780-40b4-becd-2fe0854e758e service placement] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:20.306435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4180cdc-3c00-4026-950e-1d4745442b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:20.307319 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4180cdc-3c00-4026-950e-1d4745442b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:20.307610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4180cdc-3c00-4026-950e-1d4745442b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:20.307884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4180cdc-3c00-4026-950e-1d4745442b83 admin admin] PATH_INFO: `/v3/projects/f17a6a1110a2462b98908ecbc67564f9` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:20.309194 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4180cdc-3c00-4026-950e-1d4745442b83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f17a6a1110a2462b98908ecbc67564f9)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:20.311156 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4180cdc-3c00-4026-950e-1d4745442b83 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:20.474819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:20.482632 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f4180cdc-3c00-4026-950e-1d4745442b83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f17a6a1110a2462b98908ecbc67564f9', 'request_id': 'req-f4180cdc-3c00-4026-950e-1d4745442b83'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:20.489540 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f4180cdc-3c00-4026-950e-1d4745442b83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f17a6a1110a2462b98908ecbc67564f9', 'request_id': 'req-f4180cdc-3c00-4026-950e-1d4745442b83'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:20.560645 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f4180cdc-3c00-4026-950e-1d4745442b83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f17a6a1110a2462b98908ecbc67564f9', 'request_id': 'req-f4180cdc-3c00-4026-950e-1d4745442b83'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:20.574294 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f4180cdc-3c00-4026-950e-1d4745442b83 admin admin] The token cache is being invalidate because project f17a6a1110a2462b98908ecbc67564f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:31:20.574676 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f4180cdc-3c00-4026-950e-1d4745442b83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:20.575471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-f4180cdc-3c00-4026-950e-1d4745442b83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:20.577214 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1250/2498] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:31:20 2022] DELETE /identity/v3/projects/f17a6a1110a2462b98908ecbc67564f9 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:20.581478 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b4e33-c811-435e-8169-58ceec76c972 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:20.583138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-896b4e33-c811-435e-8169-58ceec76c972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:20.608548 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b4e33-c811-435e-8169-58ceec76c972 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:20.629286 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:20.733544 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ed6e6-29aa-43dd-9c41-e994307af0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:20.736013 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ed6e6-29aa-43dd-9c41-e994307af0b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:20.736477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ed6e6-29aa-43dd-9c41-e994307af0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:20.736921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ed6e6-29aa-43dd-9c41-e994307af0b3 admin admin] PATH_INFO: `/v3/users/4c8a7a8d6b534755b83d35f8f33c4bd7` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:20.755547 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ed6e6-29aa-43dd-9c41-e994307af0b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c8a7a8d6b534755b83d35f8f33c4bd7)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:20.755547 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ed6e6-29aa-43dd-9c41-e994307af0b3 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:21.114028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-995ed6e6-29aa-43dd-9c41-e994307af0b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c8a7a8d6b534755b83d35f8f33c4bd7'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:21.121241 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-995ed6e6-29aa-43dd-9c41-e994307af0b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c8a7a8d6b534755b83d35f8f33c4bd7'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:21.121936 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-995ed6e6-29aa-43dd-9c41-e994307af0b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c8a7a8d6b534755b83d35f8f33c4bd7'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:21.122540 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-995ed6e6-29aa-43dd-9c41-e994307af0b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c8a7a8d6b534755b83d35f8f33c4bd7'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:21.135530 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-995ed6e6-29aa-43dd-9c41-e994307af0b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c8a7a8d6b534755b83d35f8f33c4bd7'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:21.311611 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1249/2499] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:31:20 2022] DELETE /identity/v3/users/4c8a7a8d6b534755b83d35f8f33c4bd7 => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:21.552938 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4180cdc-3c00-4026-950e-1d4745442b83 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:21.554781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-f4180cdc-3c00-4026-950e-1d4745442b83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:21.574190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4180cdc-3c00-4026-950e-1d4745442b83 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:21.641126 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd49363-20f2-4e21-9546-f4b374852127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:21.642425 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd49363-20f2-4e21-9546-f4b374852127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:21.642425 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd49363-20f2-4e21-9546-f4b374852127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:21.642802 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd49363-20f2-4e21-9546-f4b374852127 admin admin] PATH_INFO: `/v3/projects/2b59fd48833d405689eef891b34f5da3` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:21.644384 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd49363-20f2-4e21-9546-f4b374852127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b59fd48833d405689eef891b34f5da3)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:21.646322 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd49363-20f2-4e21-9546-f4b374852127 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:21.780673 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0fd49363-20f2-4e21-9546-f4b374852127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b59fd48833d405689eef891b34f5da3', 'request_id': 'req-0fd49363-20f2-4e21-9546-f4b374852127'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:21.795775 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0fd49363-20f2-4e21-9546-f4b374852127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b59fd48833d405689eef891b34f5da3', 'request_id': 'req-0fd49363-20f2-4e21-9546-f4b374852127'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:21.853991 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0fd49363-20f2-4e21-9546-f4b374852127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b59fd48833d405689eef891b34f5da3', 'request_id': 'req-0fd49363-20f2-4e21-9546-f4b374852127'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:21.865352 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0fd49363-20f2-4e21-9546-f4b374852127 admin admin] The token cache is being invalidate because project 2b59fd48833d405689eef891b34f5da3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:31:21.865641 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0fd49363-20f2-4e21-9546-f4b374852127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:21.866257 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0fd49363-20f2-4e21-9546-f4b374852127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:21.868344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1251/2500] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:31:21 2022] DELETE /identity/v3/projects/2b59fd48833d405689eef891b34f5da3 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:21.873267 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41da1e5-d3d5-4417-84d2-36f4dd92dcef None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:21.873767 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41da1e5-d3d5-4417-84d2-36f4dd92dcef None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:21.873890 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41da1e5-d3d5-4417-84d2-36f4dd92dcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:21.893234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-d41da1e5-d3d5-4417-84d2-36f4dd92dcef None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:21.894440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-d41da1e5-d3d5-4417-84d2-36f4dd92dcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:21.957413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1250/2501] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:31:21.964991 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a07d1c-904c-459f-ac09-71caa0c049d3 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:21.965288 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a07d1c-904c-459f-ac09-71caa0c049d3 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:21.965573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a07d1c-904c-459f-ac09-71caa0c049d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:21.985998 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-e1a07d1c-904c-459f-ac09-71caa0c049d3 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:21.987260 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-e1a07d1c-904c-459f-ac09-71caa0c049d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:21.996570 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1252/2502] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:22.000141 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41da1e5-d3d5-4417-84d2-36f4dd92dcef None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:22.010705 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41da1e5-d3d5-4417-84d2-36f4dd92dcef None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:22.015829 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622b74f-49f7-43c7-a2dc-f7c5c51d62f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:22.016499 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622b74f-49f7-43c7-a2dc-f7c5c51d62f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:22.016789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622b74f-49f7-43c7-a2dc-f7c5c51d62f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:22.017070 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622b74f-49f7-43c7-a2dc-f7c5c51d62f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:22.017838 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622b74f-49f7-43c7-a2dc-f7c5c51d62f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:31:22.018154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622b74f-49f7-43c7-a2dc-f7c5c51d62f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:22.019788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622b74f-49f7-43c7-a2dc-f7c5c51d62f9 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:22.031790 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1251/2503] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:31:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:31:22.038726 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a07d1c-904c-459f-ac09-71caa0c049d3 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:22.048102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a07d1c-904c-459f-ac09-71caa0c049d3 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:22.053087 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a106d1-f816-4f5b-bed8-3bd0afa5ab52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:22.053747 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a106d1-f816-4f5b-bed8-3bd0afa5ab52 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:22.054038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a106d1-f816-4f5b-bed8-3bd0afa5ab52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:22.054317 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a106d1-f816-4f5b-bed8-3bd0afa5ab52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:22.055568 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a106d1-f816-4f5b-bed8-3bd0afa5ab52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:22.057479 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a106d1-f816-4f5b-bed8-3bd0afa5ab52 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:22.149787 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1253/2504] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:31:22 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:22.154085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622b74f-49f7-43c7-a2dc-f7c5c51d62f9 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:22.166014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622b74f-49f7-43c7-a2dc-f7c5c51d62f9 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:22.185851 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:22.222734 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa82b0c-073c-4509-bbcf-596830a7ba64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:22.223536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa82b0c-073c-4509-bbcf-596830a7ba64 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:22.223855 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa82b0c-073c-4509-bbcf-596830a7ba64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:22.224165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa82b0c-073c-4509-bbcf-596830a7ba64 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:22.224867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa82b0c-073c-4509-bbcf-596830a7ba64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:22.226837 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa82b0c-073c-4509-bbcf-596830a7ba64 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:22.306837 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1252/2505] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:31:22 2022] POST /identity/v3/users => generated 290 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:22.310823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a106d1-f816-4f5b-bed8-3bd0afa5ab52 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:22.321229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a106d1-f816-4f5b-bed8-3bd0afa5ab52 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:22.326049 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94edf8f-bd93-4462-930b-c8dd3b6116e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:22.330174 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94edf8f-bd93-4462-930b-c8dd3b6116e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:22.330174 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94edf8f-bd93-4462-930b-c8dd3b6116e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:22.330174 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94edf8f-bd93-4462-930b-c8dd3b6116e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:22.330174 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94edf8f-bd93-4462-930b-c8dd3b6116e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:22.330174 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94edf8f-bd93-4462-930b-c8dd3b6116e5 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:22.341661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1254/2506] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:22.345609 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa82b0c-073c-4509-bbcf-596830a7ba64 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:22.356097 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa82b0c-073c-4509-bbcf-596830a7ba64 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:22.360710 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232e5fc-a769-4287-9a6d-2d32b846412d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:22.361366 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232e5fc-a769-4287-9a6d-2d32b846412d admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:22.361654 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232e5fc-a769-4287-9a6d-2d32b846412d admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:22.361931 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232e5fc-a769-4287-9a6d-2d32b846412d admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:22.362989 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1232e5fc-a769-4287-9a6d-2d32b846412d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:22.364527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1232e5fc-a769-4287-9a6d-2d32b846412d admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:22.374827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1253/2507] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:22.378823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94edf8f-bd93-4462-930b-c8dd3b6116e5 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:22.389008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94edf8f-bd93-4462-930b-c8dd3b6116e5 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:22.393642 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5be0e7-0a5a-4291-9038-ce7bebf9f061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:22.394596 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5be0e7-0a5a-4291-9038-ce7bebf9f061 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:22.394881 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5be0e7-0a5a-4291-9038-ce7bebf9f061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:22.395190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5be0e7-0a5a-4291-9038-ce7bebf9f061 admin admin] PATH_INFO: `/v3/projects/a100fa3c9d124f9089173a35a1e3691d/users/86a640cdceb34bb9bfbb7b769e797d30/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:22.409858 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5be0e7-0a5a-4291-9038-ce7bebf9f061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a100fa3c9d124f9089173a35a1e3691d, user_id=86a640cdceb34bb9bfbb7b769e797d30, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:22.412575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5be0e7-0a5a-4291-9038-ce7bebf9f061 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:22.415338 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:22.478212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1255/2508] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:22 2022] PUT /identity/v3/projects/a100fa3c9d124f9089173a35a1e3691d/users/86a640cdceb34bb9bfbb7b769e797d30/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:22.485905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9c73c-e542-4459-96bb-053f8c1f9a40 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:22.485905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9c73c-e542-4459-96bb-053f8c1f9a40 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:22.485905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9c73c-e542-4459-96bb-053f8c1f9a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:22.502981 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-13d9c73c-e542-4459-96bb-053f8c1f9a40 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:22.504488 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-13d9c73c-e542-4459-96bb-053f8c1f9a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:22.578995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1254/2509] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:22.589231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5be0e7-0a5a-4291-9038-ce7bebf9f061 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:22.591152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ca5be0e7-0a5a-4291-9038-ce7bebf9f061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:22.650961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5be0e7-0a5a-4291-9038-ce7bebf9f061 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:22.654669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a58cba-ed82-4180-ad4a-92a96ea22e63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:22.655353 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a58cba-ed82-4180-ad4a-92a96ea22e63 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:22.655645 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a58cba-ed82-4180-ad4a-92a96ea22e63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:22.655924 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a58cba-ed82-4180-ad4a-92a96ea22e63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:22.665510 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a58cba-ed82-4180-ad4a-92a96ea22e63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:22.666985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a58cba-ed82-4180-ad4a-92a96ea22e63 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:22.676487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1256/2510] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:31:28.147663 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9c73c-e542-4459-96bb-053f8c1f9a40 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:28.148395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-13d9c73c-e542-4459-96bb-053f8c1f9a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:28.164049 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9c73c-e542-4459-96bb-053f8c1f9a40 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:28.169119 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10967c53-366d-4fcc-9827-4df327fa5d69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:28.170319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10967c53-366d-4fcc-9827-4df327fa5d69 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:28.170688 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10967c53-366d-4fcc-9827-4df327fa5d69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:28.171088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10967c53-366d-4fcc-9827-4df327fa5d69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:28.175408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-10967c53-366d-4fcc-9827-4df327fa5d69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:28.189613 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10967c53-366d-4fcc-9827-4df327fa5d69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:28.191208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10967c53-366d-4fcc-9827-4df327fa5d69 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:28.208105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1255/2511] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:31:29.569893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befad3b-ab71-4870-a717-2171b39df0ac None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:29.570343 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befad3b-ab71-4870-a717-2171b39df0ac None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:29.570571 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befad3b-ab71-4870-a717-2171b39df0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:29.605925 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-7befad3b-ab71-4870-a717-2171b39df0ac None None] MFA Rules not processed for user `86a640cdceb34bb9bfbb7b769e797d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:29.607942 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-7befad3b-ab71-4870-a717-2171b39df0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:29.694340 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1257/2512] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:29.699189 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10967c53-366d-4fcc-9827-4df327fa5d69 service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:29.711430 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10967c53-366d-4fcc-9827-4df327fa5d69 service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:29.719290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16ed4a-5820-472b-9533-d9e4674c4638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:29.720087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16ed4a-5820-472b-9533-d9e4674c4638 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:29.720283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16ed4a-5820-472b-9533-d9e4674c4638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:29.720745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16ed4a-5820-472b-9533-d9e4674c4638 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:29.723340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16ed4a-5820-472b-9533-d9e4674c4638 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:29.725372 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16ed4a-5820-472b-9533-d9e4674c4638 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:29.839631 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1256/2513] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:31:29 2022] POST /identity/v3/projects => generated 379 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:29.844468 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befad3b-ab71-4870-a717-2171b39df0ac None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:29.857685 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befad3b-ab71-4870-a717-2171b39df0ac None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:29.927633 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac091b2-e531-4897-be01-7f53ff3a5a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:29.928350 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac091b2-e531-4897-be01-7f53ff3a5a1f admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:29.928638 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac091b2-e531-4897-be01-7f53ff3a5a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:29.928917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac091b2-e531-4897-be01-7f53ff3a5a1f admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:29.929615 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac091b2-e531-4897-be01-7f53ff3a5a1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:29.935176 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac091b2-e531-4897-be01-7f53ff3a5a1f admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:30.082009 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1258/2514] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:31:29 2022] POST /identity/v3/users => generated 296 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:30.085613 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16ed4a-5820-472b-9533-d9e4674c4638 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:30.098557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16ed4a-5820-472b-9533-d9e4674c4638 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:30.105412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e8548-6497-4d30-9852-a28f5ea06399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:30.107004 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e8548-6497-4d30-9852-a28f5ea06399 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:30.107476 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e8548-6497-4d30-9852-a28f5ea06399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:30.107896 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e8548-6497-4d30-9852-a28f5ea06399 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:30.108802 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e8548-6497-4d30-9852-a28f5ea06399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:30.110413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e8548-6497-4d30-9852-a28f5ea06399 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:30.129348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1257/2515] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:30.134150 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac091b2-e531-4897-be01-7f53ff3a5a1f admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:30.145366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac091b2-e531-4897-be01-7f53ff3a5a1f admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:30.150665 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38200cd5-af5e-4598-9c52-b331c5f130ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:30.151622 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38200cd5-af5e-4598-9c52-b331c5f130ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:30.151897 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38200cd5-af5e-4598-9c52-b331c5f130ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:30.152175 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38200cd5-af5e-4598-9c52-b331c5f130ae admin admin] PATH_INFO: `/v3/projects/a671f8cc8a524a68b9c74e02faa958bd/users/ad5ac186e60a4b15bc3ba9efec159ae3/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:30.177729 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:30.182740 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38200cd5-af5e-4598-9c52-b331c5f130ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=a671f8cc8a524a68b9c74e02faa958bd, user_id=ad5ac186e60a4b15bc3ba9efec159ae3, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:30.188496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38200cd5-af5e-4598-9c52-b331c5f130ae admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:30.274338 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1259/2516] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:30 2022] PUT /identity/v3/projects/a671f8cc8a524a68b9c74e02faa958bd/users/ad5ac186e60a4b15bc3ba9efec159ae3/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:30.283627 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e8548-6497-4d30-9852-a28f5ea06399 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:30.306696 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e8548-6497-4d30-9852-a28f5ea06399 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:30.397593 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:30.409747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce9d19-fe9b-45d3-89f1-6d18b3e4e229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:30.412157 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce9d19-fe9b-45d3-89f1-6d18b3e4e229 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:30.412157 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce9d19-fe9b-45d3-89f1-6d18b3e4e229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:30.412157 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce9d19-fe9b-45d3-89f1-6d18b3e4e229 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:30.412668 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce9d19-fe9b-45d3-89f1-6d18b3e4e229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:30.415487 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce9d19-fe9b-45d3-89f1-6d18b3e4e229 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:30.442453 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1258/2517] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:30.442552 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38200cd5-af5e-4598-9c52-b331c5f130ae admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:30.467195 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38200cd5-af5e-4598-9c52-b331c5f130ae admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:30.477793 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4232201-0d82-47e9-b1c0-a3639b73c26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:30.478712 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4232201-0d82-47e9-b1c0-a3639b73c26b admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:30.479027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4232201-0d82-47e9-b1c0-a3639b73c26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:30.479314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4232201-0d82-47e9-b1c0-a3639b73c26b admin admin] PATH_INFO: `/v3/projects/a671f8cc8a524a68b9c74e02faa958bd/users/ad5ac186e60a4b15bc3ba9efec159ae3/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:30.488110 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4232201-0d82-47e9-b1c0-a3639b73c26b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a671f8cc8a524a68b9c74e02faa958bd, user_id=ad5ac186e60a4b15bc3ba9efec159ae3, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:30.491669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4232201-0d82-47e9-b1c0-a3639b73c26b admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:30.509172 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1260/2518] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:30 2022] PUT /identity/v3/projects/a671f8cc8a524a68b9c74e02faa958bd/users/ad5ac186e60a4b15bc3ba9efec159ae3/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:30.932453 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce9d19-fe9b-45d3-89f1-6d18b3e4e229 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:30.935115 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-d0ce9d19-fe9b-45d3-89f1-6d18b3e4e229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:30.949403 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce9d19-fe9b-45d3-89f1-6d18b3e4e229 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:31.022580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357b575-141d-4109-a241-ff89cf9f5223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:31.025361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357b575-141d-4109-a241-ff89cf9f5223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:31.025361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357b575-141d-4109-a241-ff89cf9f5223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:31.025842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357b575-141d-4109-a241-ff89cf9f5223 admin admin] PATH_INFO: `/v3/users/38ba424297a34ddc956208e6adeca85c` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:31.027437 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357b575-141d-4109-a241-ff89cf9f5223 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38ba424297a34ddc956208e6adeca85c)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:31.029990 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357b575-141d-4109-a241-ff89cf9f5223 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:31.325310 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-7357b575-141d-4109-a241-ff89cf9f5223 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38ba424297a34ddc956208e6adeca85c'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:31.343250 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-7357b575-141d-4109-a241-ff89cf9f5223 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38ba424297a34ddc956208e6adeca85c'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:31.344028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-7357b575-141d-4109-a241-ff89cf9f5223 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38ba424297a34ddc956208e6adeca85c'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:31.344663 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-7357b575-141d-4109-a241-ff89cf9f5223 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38ba424297a34ddc956208e6adeca85c'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:31.382231 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-7357b575-141d-4109-a241-ff89cf9f5223 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38ba424297a34ddc956208e6adeca85c'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:31.563967 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1259/2519] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:31:30 2022] DELETE /identity/v3/users/38ba424297a34ddc956208e6adeca85c => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:31.568589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed1852-dfca-4e54-a28d-7f03af2e5110 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:31.569093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed1852-dfca-4e54-a28d-7f03af2e5110 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:31.569181 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed1852-dfca-4e54-a28d-7f03af2e5110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:31.604518 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-bfed1852-dfca-4e54-a28d-7f03af2e5110 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:31.609462 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-bfed1852-dfca-4e54-a28d-7f03af2e5110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:31.724284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1261/2520] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:31.772179 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357b575-141d-4109-a241-ff89cf9f5223 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:31.774900 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7357b575-141d-4109-a241-ff89cf9f5223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:31.919956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357b575-141d-4109-a241-ff89cf9f5223 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:31.926148 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf52c7-cd20-4178-9047-a6e4229daf50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:31.927166 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf52c7-cd20-4178-9047-a6e4229daf50 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:31.927564 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf52c7-cd20-4178-9047-a6e4229daf50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:31.927963 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf52c7-cd20-4178-9047-a6e4229daf50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:31.957155 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf52c7-cd20-4178-9047-a6e4229daf50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:31.958813 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf52c7-cd20-4178-9047-a6e4229daf50 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:31.984830 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1260/2521] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:31:32.631912 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed1852-dfca-4e54-a28d-7f03af2e5110 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:32.636334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-bfed1852-dfca-4e54-a28d-7f03af2e5110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:32.660131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed1852-dfca-4e54-a28d-7f03af2e5110 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:32.675214 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e03c7-d528-4053-a85d-740b21690d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:32.676124 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e03c7-d528-4053-a85d-740b21690d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:32.676411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e03c7-d528-4053-a85d-740b21690d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:32.676693 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e03c7-d528-4053-a85d-740b21690d6b admin admin] PATH_INFO: `/v3/projects/e7fc8503ca8a423c9776eb92c9c79d53` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:32.681529 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e03c7-d528-4053-a85d-740b21690d6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7fc8503ca8a423c9776eb92c9c79d53)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:32.688210 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e03c7-d528-4053-a85d-740b21690d6b admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:32.691281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf52c7-cd20-4178-9047-a6e4229daf50 service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:32.694507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-1baf52c7-cd20-4178-9047-a6e4229daf50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:32.729470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf52c7-cd20-4178-9047-a6e4229daf50 service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:32.734305 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bf0cf-b7cc-485e-82d6-823b40bf85b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:32.735349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bf0cf-b7cc-485e-82d6-823b40bf85b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:32.735645 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bf0cf-b7cc-485e-82d6-823b40bf85b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:32.735926 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bf0cf-b7cc-485e-82d6-823b40bf85b8 admin admin] PATH_INFO: `/v3/users/ad5452341ccc40bc94b38369dae4f2b8` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:32.737184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418bf0cf-b7cc-485e-82d6-823b40bf85b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad5452341ccc40bc94b38369dae4f2b8)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:32.739812 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418bf0cf-b7cc-485e-82d6-823b40bf85b8 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:32.748729 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:32.829108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2b5e03c7-d528-4053-a85d-740b21690d6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7fc8503ca8a423c9776eb92c9c79d53', 'request_id': 'req-2b5e03c7-d528-4053-a85d-740b21690d6b'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:32.843751 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2b5e03c7-d528-4053-a85d-740b21690d6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7fc8503ca8a423c9776eb92c9c79d53', 'request_id': 'req-2b5e03c7-d528-4053-a85d-740b21690d6b'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:32.971507 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2b5e03c7-d528-4053-a85d-740b21690d6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7fc8503ca8a423c9776eb92c9c79d53', 'request_id': 'req-2b5e03c7-d528-4053-a85d-740b21690d6b'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:32.992040 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2b5e03c7-d528-4053-a85d-740b21690d6b admin admin] The token cache is being invalidate because project e7fc8503ca8a423c9776eb92c9c79d53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:31:32.993603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2b5e03c7-d528-4053-a85d-740b21690d6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:32.995647 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2b5e03c7-d528-4053-a85d-740b21690d6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:32.999023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1262/2522] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:31:32 2022] DELETE /identity/v3/projects/e7fc8503ca8a423c9776eb92c9c79d53 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:33.003718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e03c7-d528-4053-a85d-740b21690d6b admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:33.003916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-418bf0cf-b7cc-485e-82d6-823b40bf85b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad5452341ccc40bc94b38369dae4f2b8'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:33.007493 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-2b5e03c7-d528-4053-a85d-740b21690d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:33.014906 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-418bf0cf-b7cc-485e-82d6-823b40bf85b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad5452341ccc40bc94b38369dae4f2b8'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:33.015904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-418bf0cf-b7cc-485e-82d6-823b40bf85b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad5452341ccc40bc94b38369dae4f2b8'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:33.017043 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-418bf0cf-b7cc-485e-82d6-823b40bf85b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad5452341ccc40bc94b38369dae4f2b8'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:33.036552 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-418bf0cf-b7cc-485e-82d6-823b40bf85b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad5452341ccc40bc94b38369dae4f2b8'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:33.108271 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e03c7-d528-4053-a85d-740b21690d6b admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:33.113700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a829bd-08e2-440c-834c-5fe527872b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:33.114593 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a829bd-08e2-440c-834c-5fe527872b72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:33.115023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a829bd-08e2-440c-834c-5fe527872b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:33.115265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a829bd-08e2-440c-834c-5fe527872b72 admin admin] PATH_INFO: `/v3/users/14185027c8884ee28a1ded4464e418d6` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:33.121079 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a829bd-08e2-440c-834c-5fe527872b72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14185027c8884ee28a1ded4464e418d6)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:33.126533 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a829bd-08e2-440c-834c-5fe527872b72 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:33.129403 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:33.158618 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1261/2523] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:31:32 2022] DELETE /identity/v3/users/ad5452341ccc40bc94b38369dae4f2b8 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:33.355376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-40a829bd-08e2-440c-834c-5fe527872b72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14185027c8884ee28a1ded4464e418d6'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:33.367052 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-40a829bd-08e2-440c-834c-5fe527872b72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14185027c8884ee28a1ded4464e418d6'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:33.369156 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-40a829bd-08e2-440c-834c-5fe527872b72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14185027c8884ee28a1ded4464e418d6'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:33.370044 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-40a829bd-08e2-440c-834c-5fe527872b72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14185027c8884ee28a1ded4464e418d6'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:33.383798 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-40a829bd-08e2-440c-834c-5fe527872b72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14185027c8884ee28a1ded4464e418d6'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:33.494786 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1263/2524] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:31:33 2022] DELETE /identity/v3/users/14185027c8884ee28a1ded4464e418d6 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:33.498734 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bf0cf-b7cc-485e-82d6-823b40bf85b8 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:33.500915 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-418bf0cf-b7cc-485e-82d6-823b40bf85b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:33.612502 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bf0cf-b7cc-485e-82d6-823b40bf85b8 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:33.675715 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f97b19-155f-47ae-b920-7469b558c8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:33.676597 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f97b19-155f-47ae-b920-7469b558c8e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:33.676942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f97b19-155f-47ae-b920-7469b558c8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:33.677228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f97b19-155f-47ae-b920-7469b558c8e7 admin admin] PATH_INFO: `/v3/projects/8d0795f7c61d497c9b25c7e17ad08744` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:33.679083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f97b19-155f-47ae-b920-7469b558c8e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d0795f7c61d497c9b25c7e17ad08744)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:33.681087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f97b19-155f-47ae-b920-7469b558c8e7 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:33.925271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-23f97b19-155f-47ae-b920-7469b558c8e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d0795f7c61d497c9b25c7e17ad08744', 'request_id': 'req-23f97b19-155f-47ae-b920-7469b558c8e7'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:33.939541 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-23f97b19-155f-47ae-b920-7469b558c8e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d0795f7c61d497c9b25c7e17ad08744', 'request_id': 'req-23f97b19-155f-47ae-b920-7469b558c8e7'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:33.943077 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-23f97b19-155f-47ae-b920-7469b558c8e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d0795f7c61d497c9b25c7e17ad08744', 'request_id': 'req-23f97b19-155f-47ae-b920-7469b558c8e7'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:34.044631 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-23f97b19-155f-47ae-b920-7469b558c8e7 admin admin] The token cache is being invalidate because project 8d0795f7c61d497c9b25c7e17ad08744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:31:34.044631 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-23f97b19-155f-47ae-b920-7469b558c8e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:34.044631 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-23f97b19-155f-47ae-b920-7469b558c8e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:34.045921 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1262/2525] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:31:33 2022] DELETE /identity/v3/projects/8d0795f7c61d497c9b25c7e17ad08744 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:34.050882 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f36532-c5b3-41f4-8463-3a8d3302ca41 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:34.051373 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f36532-c5b3-41f4-8463-3a8d3302ca41 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:34.051576 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f36532-c5b3-41f4-8463-3a8d3302ca41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:34.073845 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-45f36532-c5b3-41f4-8463-3a8d3302ca41 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:34.081847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-45f36532-c5b3-41f4-8463-3a8d3302ca41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:34.161775 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1264/2526] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:34.170712 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce435f85-48c2-4ba7-9734-52fe6c7877bc None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:34.170712 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce435f85-48c2-4ba7-9734-52fe6c7877bc None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:34.170712 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce435f85-48c2-4ba7-9734-52fe6c7877bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:34.202083 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-ce435f85-48c2-4ba7-9734-52fe6c7877bc None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:34.203688 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-ce435f85-48c2-4ba7-9734-52fe6c7877bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:34.212721 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1263/2527] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:34.217611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f36532-c5b3-41f4-8463-3a8d3302ca41 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:34.228859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f36532-c5b3-41f4-8463-3a8d3302ca41 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:34.238003 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb92e4-6481-4956-9d3f-b345c50e5581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:34.238777 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb92e4-6481-4956-9d3f-b345c50e5581 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:34.239121 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb92e4-6481-4956-9d3f-b345c50e5581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:34.240314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb92e4-6481-4956-9d3f-b345c50e5581 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:34.240314 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb92e4-6481-4956-9d3f-b345c50e5581 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:31:34.240622 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb92e4-6481-4956-9d3f-b345c50e5581 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:34.242287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb92e4-6481-4956-9d3f-b345c50e5581 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:34.255289 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1265/2528] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:31:34.259801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce435f85-48c2-4ba7-9734-52fe6c7877bc None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:34.269192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce435f85-48c2-4ba7-9734-52fe6c7877bc None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:34.273861 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f4c65-b080-4b9f-b226-ddfff3ae19df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:34.274522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f4c65-b080-4b9f-b226-ddfff3ae19df admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:34.274813 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f4c65-b080-4b9f-b226-ddfff3ae19df admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:34.282755 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f4c65-b080-4b9f-b226-ddfff3ae19df admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:34.282755 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f4c65-b080-4b9f-b226-ddfff3ae19df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:34.282755 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f4c65-b080-4b9f-b226-ddfff3ae19df admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:34.355292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb92e4-6481-4956-9d3f-b345c50e5581 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:34.357133 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-68bb92e4-6481-4956-9d3f-b345c50e5581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:34.372627 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb92e4-6481-4956-9d3f-b345c50e5581 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:34.379219 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b1ed8-bcc5-4e49-819c-d41072fa49f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:34.379827 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b1ed8-bcc5-4e49-819c-d41072fa49f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:34.380258 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b1ed8-bcc5-4e49-819c-d41072fa49f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:34.380550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b1ed8-bcc5-4e49-819c-d41072fa49f0 admin admin] PATH_INFO: `/v3/projects/d957cdcf5fb34d6c951449d892cc00de` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:34.381776 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b1ed8-bcc5-4e49-819c-d41072fa49f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d957cdcf5fb34d6c951449d892cc00de)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:34.383829 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b1ed8-bcc5-4e49-819c-d41072fa49f0 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:34.427235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1264/2529] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:31:34 2022] POST /identity/v3/projects => generated 403 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:34.432097 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f4c65-b080-4b9f-b226-ddfff3ae19df admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:34.502111 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f4c65-b080-4b9f-b226-ddfff3ae19df admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:34.506974 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93bcdc2-737c-4b8a-93ae-4e088bc8a9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:34.507638 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bcdc2-737c-4b8a-93ae-4e088bc8a9f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:34.507970 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bcdc2-737c-4b8a-93ae-4e088bc8a9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:34.508256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bcdc2-737c-4b8a-93ae-4e088bc8a9f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:34.509110 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93bcdc2-737c-4b8a-93ae-4e088bc8a9f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:34.511350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93bcdc2-737c-4b8a-93ae-4e088bc8a9f1 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:34.544958 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-973b1ed8-bcc5-4e49-819c-d41072fa49f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd957cdcf5fb34d6c951449d892cc00de', 'request_id': 'req-973b1ed8-bcc5-4e49-819c-d41072fa49f0'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:34.557098 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-973b1ed8-bcc5-4e49-819c-d41072fa49f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd957cdcf5fb34d6c951449d892cc00de', 'request_id': 'req-973b1ed8-bcc5-4e49-819c-d41072fa49f0'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:34.649385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1265/2530] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:31:34 2022] POST /identity/v3/users => generated 302 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:34.657460 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93bcdc2-737c-4b8a-93ae-4e088bc8a9f1 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:34.665708 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93bcdc2-737c-4b8a-93ae-4e088bc8a9f1 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:34.699527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-973b1ed8-bcc5-4e49-819c-d41072fa49f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd957cdcf5fb34d6c951449d892cc00de', 'request_id': 'req-973b1ed8-bcc5-4e49-819c-d41072fa49f0'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:34.720248 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-973b1ed8-bcc5-4e49-819c-d41072fa49f0 admin admin] The token cache is being invalidate because project d957cdcf5fb34d6c951449d892cc00de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:31:34.720545 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-973b1ed8-bcc5-4e49-819c-d41072fa49f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:34.722388 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-973b1ed8-bcc5-4e49-819c-d41072fa49f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:34.725714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1266/2531] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:31:34 2022] DELETE /identity/v3/projects/d957cdcf5fb34d6c951449d892cc00de => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:31:34.733472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3c679-a021-4254-a2fa-be174ae6c405 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:34.734070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3c679-a021-4254-a2fa-be174ae6c405 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:34.734538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3c679-a021-4254-a2fa-be174ae6c405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:34.735691 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c16d5-e41a-46bc-8e13-1a9ba57ff8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:34.736380 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c16d5-e41a-46bc-8e13-1a9ba57ff8fa admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:34.736728 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c16d5-e41a-46bc-8e13-1a9ba57ff8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:34.736956 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c16d5-e41a-46bc-8e13-1a9ba57ff8fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:34.738054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c16d5-e41a-46bc-8e13-1a9ba57ff8fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:34.739869 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c16d5-e41a-46bc-8e13-1a9ba57ff8fa admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:34.754429 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1266/2532] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:34.755525 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c16d5-e41a-46bc-8e13-1a9ba57ff8fa admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:34.757330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-1b2c16d5-e41a-46bc-8e13-1a9ba57ff8fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:34.757608 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-d0c3c679-a021-4254-a2fa-be174ae6c405 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:34.759753 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d0c3c679-a021-4254-a2fa-be174ae6c405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:34.769598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1267/2533] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:34.771607 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c16d5-e41a-46bc-8e13-1a9ba57ff8fa admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:34.776813 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336e2fc-b5fe-44a3-9182-cd813f15a08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:34.777515 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336e2fc-b5fe-44a3-9182-cd813f15a08e admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:34.777801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336e2fc-b5fe-44a3-9182-cd813f15a08e admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:34.778089 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336e2fc-b5fe-44a3-9182-cd813f15a08e admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:34.778839 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336e2fc-b5fe-44a3-9182-cd813f15a08e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:34.779697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744f892-a3ad-4d1b-8790-bbfc274ac7bc None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:34.780014 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744f892-a3ad-4d1b-8790-bbfc274ac7bc None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:34.780287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744f892-a3ad-4d1b-8790-bbfc274ac7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:34.780753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336e2fc-b5fe-44a3-9182-cd813f15a08e admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:34.792110 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1267/2534] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:34.796137 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336e2fc-b5fe-44a3-9182-cd813f15a08e admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:34.799113 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-7744f892-a3ad-4d1b-8790-bbfc274ac7bc None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:34.800310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-7744f892-a3ad-4d1b-8790-bbfc274ac7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:34.806689 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336e2fc-b5fe-44a3-9182-cd813f15a08e admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:34.810131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1268/2535] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:34.811393 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056dae4-966e-4a0a-8005-db3847610613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:34.812286 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056dae4-966e-4a0a-8005-db3847610613 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:34.812579 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056dae4-966e-4a0a-8005-db3847610613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:34.812862 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056dae4-966e-4a0a-8005-db3847610613 admin admin] PATH_INFO: `/v3/projects/da9c33b4686f4655927d30cc24950141/users/69a7493862484a078dd9b33d97068149/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:34.814382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744f892-a3ad-4d1b-8790-bbfc274ac7bc None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:34.836829 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744f892-a3ad-4d1b-8790-bbfc274ac7bc None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:34.836937 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056dae4-966e-4a0a-8005-db3847610613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da9c33b4686f4655927d30cc24950141, user_id=69a7493862484a078dd9b33d97068149, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:34.837027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6181d-bed7-414c-93de-bfbc77c85d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:34.837027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6181d-bed7-414c-93de-bfbc77c85d7d admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:34.837027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6181d-bed7-414c-93de-bfbc77c85d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:34.837027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6181d-bed7-414c-93de-bfbc77c85d7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:34.837390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056dae4-966e-4a0a-8005-db3847610613 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:34.837487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6181d-bed7-414c-93de-bfbc77c85d7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:31:34.837487 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6181d-bed7-414c-93de-bfbc77c85d7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:34.838022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6181d-bed7-414c-93de-bfbc77c85d7d admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:34.850341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1269/2536] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:31:34.855111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6181d-bed7-414c-93de-bfbc77c85d7d admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:34.865211 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6181d-bed7-414c-93de-bfbc77c85d7d admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:34.870538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf88ea4-0e43-4522-b8d0-0166b3493acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:34.871435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf88ea4-0e43-4522-b8d0-0166b3493acb admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:34.871830 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf88ea4-0e43-4522-b8d0-0166b3493acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:34.872176 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf88ea4-0e43-4522-b8d0-0166b3493acb admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:34.872930 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf88ea4-0e43-4522-b8d0-0166b3493acb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:34.874935 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf88ea4-0e43-4522-b8d0-0166b3493acb admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:34.968610 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1268/2537] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:34 2022] PUT /identity/v3/projects/da9c33b4686f4655927d30cc24950141/users/69a7493862484a078dd9b33d97068149/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:31:34.973512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88a326-a112-4b67-b1dd-a66d484b4182 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:34.974501 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88a326-a112-4b67-b1dd-a66d484b4182 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:34.974788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88a326-a112-4b67-b1dd-a66d484b4182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:35.002483 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-dd88a326-a112-4b67-b1dd-a66d484b4182 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:35.004588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-dd88a326-a112-4b67-b1dd-a66d484b4182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:35.034416 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1270/2538] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:31:34 2022] POST /identity/v3/projects => generated 369 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:31:35.038792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf88ea4-0e43-4522-b8d0-0166b3493acb admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:35.076528 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1269/2539] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:31:35.086494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88a326-a112-4b67-b1dd-a66d484b4182 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:35.088125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-dd88a326-a112-4b67-b1dd-a66d484b4182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:35.145972 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf88ea4-0e43-4522-b8d0-0166b3493acb admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:35.157255 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7cb57-4508-4767-8510-25ad714a38cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:35.157923 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7cb57-4508-4767-8510-25ad714a38cd admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:35.158214 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7cb57-4508-4767-8510-25ad714a38cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:35.158681 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7cb57-4508-4767-8510-25ad714a38cd admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:35.159424 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7cb57-4508-4767-8510-25ad714a38cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:35.161401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7cb57-4508-4767-8510-25ad714a38cd admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:35.181835 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88a326-a112-4b67-b1dd-a66d484b4182 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:35.258943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f151d7-8340-45ac-b9de-cf5d6296a3fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:35.259748 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f151d7-8340-45ac-b9de-cf5d6296a3fa service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:35.260210 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f151d7-8340-45ac-b9de-cf5d6296a3fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:35.260657 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f151d7-8340-45ac-b9de-cf5d6296a3fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:35.271011 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f151d7-8340-45ac-b9de-cf5d6296a3fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:35.272511 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f151d7-8340-45ac-b9de-cf5d6296a3fa service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:35.283385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1270/2540] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:31:35.295997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1271/2541] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:31:35 2022] POST /identity/v3/users => generated 285 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:35.297692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f151d7-8340-45ac-b9de-cf5d6296a3fa service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:35.308800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f151d7-8340-45ac-b9de-cf5d6296a3fa service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:35.317237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5961e-889c-434f-a224-7e8c9e4e71bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:35.317237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5961e-889c-434f-a224-7e8c9e4e71bb admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:35.317563 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5961e-889c-434f-a224-7e8c9e4e71bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:35.317972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5961e-889c-434f-a224-7e8c9e4e71bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:35.319305 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5961e-889c-434f-a224-7e8c9e4e71bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:35.321264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5961e-889c-434f-a224-7e8c9e4e71bb admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:35.329582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:35.339185 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1271/2542] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:35.341017 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7cb57-4508-4767-8510-25ad714a38cd admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:35.353155 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7cb57-4508-4767-8510-25ad714a38cd admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:35.358292 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fffb2f-23ba-409e-af03-67fcd88a0165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:35.359033 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fffb2f-23ba-409e-af03-67fcd88a0165 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:35.359321 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fffb2f-23ba-409e-af03-67fcd88a0165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:35.359598 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fffb2f-23ba-409e-af03-67fcd88a0165 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:35.360372 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fffb2f-23ba-409e-af03-67fcd88a0165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:35.361990 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fffb2f-23ba-409e-af03-67fcd88a0165 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:35.373951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1272/2543] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:35.379322 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5961e-889c-434f-a224-7e8c9e4e71bb admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:35.390901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5961e-889c-434f-a224-7e8c9e4e71bb admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:35.396305 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3cadc-8298-4500-81dd-488190915fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:35.397249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3cadc-8298-4500-81dd-488190915fd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:35.397610 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3cadc-8298-4500-81dd-488190915fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:35.398439 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3cadc-8298-4500-81dd-488190915fd0 admin admin] PATH_INFO: `/v3/projects/d3e481e5158b499c8da9dd6fb2da2a9f/users/28c79a89b0c44cd396ce572fe03d72ca/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:35.422827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3cadc-8298-4500-81dd-488190915fd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3e481e5158b499c8da9dd6fb2da2a9f, user_id=28c79a89b0c44cd396ce572fe03d72ca, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:35.426043 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3cadc-8298-4500-81dd-488190915fd0 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:35.620338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1272/2544] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:35 2022] PUT /identity/v3/projects/d3e481e5158b499c8da9dd6fb2da2a9f/users/28c79a89b0c44cd396ce572fe03d72ca/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:31:35.632907 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbe2d8-a497-41fb-977b-3b419c3e70cd None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:35.633200 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbe2d8-a497-41fb-977b-3b419c3e70cd None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:35.633480 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbe2d8-a497-41fb-977b-3b419c3e70cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:35.667622 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-49cbe2d8-a497-41fb-977b-3b419c3e70cd None None] MFA Rules not processed for user `28c79a89b0c44cd396ce572fe03d72ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:35.669969 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-49cbe2d8-a497-41fb-977b-3b419c3e70cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:35.755429 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1273/2545] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:35 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:35.756750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3cadc-8298-4500-81dd-488190915fd0 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:35.850385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3cadc-8298-4500-81dd-488190915fd0 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:35.858249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71723a4e-fbf4-4ab1-bdf1-2890aaa1d0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:35.859001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71723a4e-fbf4-4ab1-bdf1-2890aaa1d0cb admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:35.859300 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71723a4e-fbf4-4ab1-bdf1-2890aaa1d0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:35.859591 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71723a4e-fbf4-4ab1-bdf1-2890aaa1d0cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:35.860295 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71723a4e-fbf4-4ab1-bdf1-2890aaa1d0cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:35.862234 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71723a4e-fbf4-4ab1-bdf1-2890aaa1d0cb admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:35.968635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:36.019599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1273/2546] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:31:35 2022] POST /identity/v3/projects => generated 369 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:36.020942 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbe2d8-a497-41fb-977b-3b419c3e70cd None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:36.036678 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbe2d8-a497-41fb-977b-3b419c3e70cd None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:36.103077 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816e116-4cf6-4a05-a238-fb4db7743e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:36.103949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816e116-4cf6-4a05-a238-fb4db7743e20 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:36.104313 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816e116-4cf6-4a05-a238-fb4db7743e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:36.104948 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816e116-4cf6-4a05-a238-fb4db7743e20 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:36.105523 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f816e116-4cf6-4a05-a238-fb4db7743e20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:36.107541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f816e116-4cf6-4a05-a238-fb4db7743e20 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:36.248659 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1274/2547] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:31:36 2022] POST /identity/v3/users => generated 291 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:36.252892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71723a4e-fbf4-4ab1-bdf1-2890aaa1d0cb admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:36.263817 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71723a4e-fbf4-4ab1-bdf1-2890aaa1d0cb admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:36.268653 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29892450-483e-479c-aa2b-a5df32dd056b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:36.269325 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29892450-483e-479c-aa2b-a5df32dd056b admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:36.269629 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29892450-483e-479c-aa2b-a5df32dd056b admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:36.269899 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29892450-483e-479c-aa2b-a5df32dd056b admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:36.270634 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29892450-483e-479c-aa2b-a5df32dd056b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:36.272349 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29892450-483e-479c-aa2b-a5df32dd056b admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:36.283824 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1274/2548] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:36.288060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816e116-4cf6-4a05-a238-fb4db7743e20 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:36.298660 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816e116-4cf6-4a05-a238-fb4db7743e20 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:36.303285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fdc97-d854-4a51-84c0-ce9fa1fc83eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:36.304155 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fdc97-d854-4a51-84c0-ce9fa1fc83eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:36.304440 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fdc97-d854-4a51-84c0-ce9fa1fc83eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:36.304741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fdc97-d854-4a51-84c0-ce9fa1fc83eb admin admin] PATH_INFO: `/v3/projects/73660378e226452db04380659e53d969/users/709f8e6f6d154ad4990122378f5d3dda/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:36.320803 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5fdc97-d854-4a51-84c0-ce9fa1fc83eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=73660378e226452db04380659e53d969, user_id=709f8e6f6d154ad4990122378f5d3dda, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:36.323503 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5fdc97-d854-4a51-84c0-ce9fa1fc83eb admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:36.543394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1275/2549] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:36 2022] PUT /identity/v3/projects/73660378e226452db04380659e53d969/users/709f8e6f6d154ad4990122378f5d3dda/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:36.544639 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29892450-483e-479c-aa2b-a5df32dd056b admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:36.561336 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29892450-483e-479c-aa2b-a5df32dd056b admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:36.641375 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c37b2d-9b55-4117-920f-bb03656d0ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:36.642343 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c37b2d-9b55-4117-920f-bb03656d0ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:36.642804 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c37b2d-9b55-4117-920f-bb03656d0ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:36.643270 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c37b2d-9b55-4117-920f-bb03656d0ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:36.644192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c37b2d-9b55-4117-920f-bb03656d0ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:36.646430 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c37b2d-9b55-4117-920f-bb03656d0ee3 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:36.662518 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1275/2550] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:36.663757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fdc97-d854-4a51-84c0-ce9fa1fc83eb admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:36.675315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fdc97-d854-4a51-84c0-ce9fa1fc83eb admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:36.680960 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf750420-ec12-4478-aadd-f79af78fee42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:36.684496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf750420-ec12-4478-aadd-f79af78fee42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:36.684496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf750420-ec12-4478-aadd-f79af78fee42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:36.684496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf750420-ec12-4478-aadd-f79af78fee42 admin admin] PATH_INFO: `/v3/projects/73660378e226452db04380659e53d969/users/709f8e6f6d154ad4990122378f5d3dda/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:36.685092 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf750420-ec12-4478-aadd-f79af78fee42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73660378e226452db04380659e53d969, user_id=709f8e6f6d154ad4990122378f5d3dda, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:36.687500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf750420-ec12-4478-aadd-f79af78fee42 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:36.699450 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1276/2551] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:36 2022] PUT /identity/v3/projects/73660378e226452db04380659e53d969/users/709f8e6f6d154ad4990122378f5d3dda/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:36.709352 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1deae8-07eb-4298-a848-735955e93d38 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:36.709653 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1deae8-07eb-4298-a848-735955e93d38 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:36.709936 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1deae8-07eb-4298-a848-735955e93d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:36.743868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-af1deae8-07eb-4298-a848-735955e93d38 None None] MFA Rules not processed for user `709f8e6f6d154ad4990122378f5d3dda`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:36.745522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-af1deae8-07eb-4298-a848-735955e93d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:36.828963 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1276/2552] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:36 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:36.844118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf750420-ec12-4478-aadd-f79af78fee42 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:36.846109 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-cf750420-ec12-4478-aadd-f79af78fee42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:36.910879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf750420-ec12-4478-aadd-f79af78fee42 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:36.914733 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1824689-1983-4008-8782-f037d96a4a49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:36.915432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1824689-1983-4008-8782-f037d96a4a49 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:36.915722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1824689-1983-4008-8782-f037d96a4a49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:36.916004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1824689-1983-4008-8782-f037d96a4a49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:36.926454 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1824689-1983-4008-8782-f037d96a4a49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:36.930364 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1824689-1983-4008-8782-f037d96a4a49 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:36.939478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1277/2553] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:36 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:31:37.890654 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bc60b-6932-44f6-b327-f99868f17e16 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:37.891238 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bc60b-6932-44f6-b327-f99868f17e16 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:37.891238 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bc60b-6932-44f6-b327-f99868f17e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:37.918476 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-980bc60b-6932-44f6-b327-f99868f17e16 None None] MFA Rules not processed for user `ad5ac186e60a4b15bc3ba9efec159ae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:37.919747 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-980bc60b-6932-44f6-b327-f99868f17e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:37.970484 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b682609-7ccc-4d5a-9f5d-050ba085172e None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:37.971015 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b682609-7ccc-4d5a-9f5d-050ba085172e None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:37.971117 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b682609-7ccc-4d5a-9f5d-050ba085172e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:37.996861 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1277/2554] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:37 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:38.010404 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-0b682609-7ccc-4d5a-9f5d-050ba085172e None None] MFA Rules not processed for user `69a7493862484a078dd9b33d97068149`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:38.011093 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980bc60b-6932-44f6-b327-f99868f17e16 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:38.011685 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-0b682609-7ccc-4d5a-9f5d-050ba085172e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:38.012592 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-980bc60b-6932-44f6-b327-f99868f17e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:38.075610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1278/2555] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:38.080018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b682609-7ccc-4d5a-9f5d-050ba085172e None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:38.083669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980bc60b-6932-44f6-b327-f99868f17e16 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:38.089865 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd1290-e9cc-4d8c-92ab-9a33fc096e11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:38.090584 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd1290-e9cc-4d8c-92ab-9a33fc096e11 service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:38.090875 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd1290-e9cc-4d8c-92ab-9a33fc096e11 service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:38.091344 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd1290-e9cc-4d8c-92ab-9a33fc096e11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:38.093121 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-f4cd1290-e9cc-4d8c-92ab-9a33fc096e11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:38.143320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b682609-7ccc-4d5a-9f5d-050ba085172e None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:38.147865 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d951c-7a97-43ba-8848-0327367b9119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:38.149082 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d951c-7a97-43ba-8848-0327367b9119 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:38.149376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d951c-7a97-43ba-8848-0327367b9119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:38.149648 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d951c-7a97-43ba-8848-0327367b9119 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:38.150333 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37d951c-7a97-43ba-8848-0327367b9119 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:38.151726 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd1290-e9cc-4d8c-92ab-9a33fc096e11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:38.152684 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37d951c-7a97-43ba-8848-0327367b9119 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:38.153367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd1290-e9cc-4d8c-92ab-9a33fc096e11 service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:38.173478 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1278/2556] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:31:38.297341 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1279/2557] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:31:38 2022] POST /identity/v3/projects => generated 405 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:31:38.307782 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd1290-e9cc-4d8c-92ab-9a33fc096e11 service nova] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:38.322037 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd1290-e9cc-4d8c-92ab-9a33fc096e11 service nova] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:38.389372 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5acfa6-7f74-425b-bcae-821d63575e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:38.390053 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5acfa6-7f74-425b-bcae-821d63575e4d admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:38.390342 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5acfa6-7f74-425b-bcae-821d63575e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:38.390624 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5acfa6-7f74-425b-bcae-821d63575e4d admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:38.391371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5acfa6-7f74-425b-bcae-821d63575e4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:38.393304 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5acfa6-7f74-425b-bcae-821d63575e4d admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:38.505768 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1279/2558] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:31:38 2022] POST /identity/v3/users => generated 309 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:31:38.509920 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d951c-7a97-43ba-8848-0327367b9119 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:38.520206 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d951c-7a97-43ba-8848-0327367b9119 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:38.524758 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b083e-9442-404e-bc41-9e70389bd460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:38.525407 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b083e-9442-404e-bc41-9e70389bd460 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:38.525693 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b083e-9442-404e-bc41-9e70389bd460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:38.525970 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b083e-9442-404e-bc41-9e70389bd460 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:38.526693 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b083e-9442-404e-bc41-9e70389bd460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:38.528330 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b083e-9442-404e-bc41-9e70389bd460 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:38.538604 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1280/2559] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:38.549686 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5acfa6-7f74-425b-bcae-821d63575e4d admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:38.560180 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5acfa6-7f74-425b-bcae-821d63575e4d admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:38.564764 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6678dd-010e-409e-afda-6d10937b485c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:38.565648 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6678dd-010e-409e-afda-6d10937b485c admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:38.565937 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6678dd-010e-409e-afda-6d10937b485c admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:38.566215 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6678dd-010e-409e-afda-6d10937b485c admin admin] PATH_INFO: `/v3/projects/13798ae7abba44a8893cd5401d1684f0/users/b20a52f21c3c4c5d9ab7019d1965d73f/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:38.583338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6678dd-010e-409e-afda-6d10937b485c admin admin] RBAC: Authorizing `identity:create_grant(project_id=13798ae7abba44a8893cd5401d1684f0, user_id=b20a52f21c3c4c5d9ab7019d1965d73f, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:38.592961 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6678dd-010e-409e-afda-6d10937b485c admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:38.660660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:38.745850 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1280/2560] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:38 2022] PUT /identity/v3/projects/13798ae7abba44a8893cd5401d1684f0/users/b20a52f21c3c4c5d9ab7019d1965d73f/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:38.750741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b083e-9442-404e-bc41-9e70389bd460 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:38.760702 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b083e-9442-404e-bc41-9e70389bd460 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:38.829380 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850f815-a678-4a06-9757-d29fd9c22414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:38.830050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850f815-a678-4a06-9757-d29fd9c22414 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:38.830342 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850f815-a678-4a06-9757-d29fd9c22414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:38.830623 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850f815-a678-4a06-9757-d29fd9c22414 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:38.831390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850f815-a678-4a06-9757-d29fd9c22414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:38.833024 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850f815-a678-4a06-9757-d29fd9c22414 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:38.847721 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1281/2561] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:38.848839 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6678dd-010e-409e-afda-6d10937b485c admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:38.860026 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6678dd-010e-409e-afda-6d10937b485c admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:38.865107 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c604ea-5cca-43c7-ba80-b68976e1c3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:38.866008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c604ea-5cca-43c7-ba80-b68976e1c3c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:38.866294 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c604ea-5cca-43c7-ba80-b68976e1c3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:38.866574 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c604ea-5cca-43c7-ba80-b68976e1c3c0 admin admin] PATH_INFO: `/v3/projects/13798ae7abba44a8893cd5401d1684f0/users/b20a52f21c3c4c5d9ab7019d1965d73f/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:38.869222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c604ea-5cca-43c7-ba80-b68976e1c3c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13798ae7abba44a8893cd5401d1684f0, user_id=b20a52f21c3c4c5d9ab7019d1965d73f, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:38.871910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c604ea-5cca-43c7-ba80-b68976e1c3c0 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:38.881902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1281/2562] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:38 2022] PUT /identity/v3/projects/13798ae7abba44a8893cd5401d1684f0/users/b20a52f21c3c4c5d9ab7019d1965d73f/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:39.494571 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:40.789634 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbdab6-a6f3-43f1-9925-314efd3ad1b5 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:40.790212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbdab6-a6f3-43f1-9925-314efd3ad1b5 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:40.790212 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbdab6-a6f3-43f1-9925-314efd3ad1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:40.853604 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-62dbdab6-a6f3-43f1-9925-314efd3ad1b5 None None] MFA Rules not processed for user `b20a52f21c3c4c5d9ab7019d1965d73f`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:40.854842 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-62dbdab6-a6f3-43f1-9925-314efd3ad1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:40.938321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c604ea-5cca-43c7-ba80-b68976e1c3c0 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:40.966962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1282/2563] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:40 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:41.036144 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c604ea-5cca-43c7-ba80-b68976e1c3c0 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:41.043158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22faf3f3-f186-4a41-bab3-7d93f780c0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:41.044439 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22faf3f3-f186-4a41-bab3-7d93f780c0f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:41.044612 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22faf3f3-f186-4a41-bab3-7d93f780c0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:41.044991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22faf3f3-f186-4a41-bab3-7d93f780c0f2 admin admin] PATH_INFO: `/v3/users/28c79a89b0c44cd396ce572fe03d72ca` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:41.047663 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22faf3f3-f186-4a41-bab3-7d93f780c0f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28c79a89b0c44cd396ce572fe03d72ca)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:41.049594 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22faf3f3-f186-4a41-bab3-7d93f780c0f2 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:41.250293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-22faf3f3-f186-4a41-bab3-7d93f780c0f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28c79a89b0c44cd396ce572fe03d72ca'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:41.258792 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-22faf3f3-f186-4a41-bab3-7d93f780c0f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28c79a89b0c44cd396ce572fe03d72ca'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:41.259692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-22faf3f3-f186-4a41-bab3-7d93f780c0f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28c79a89b0c44cd396ce572fe03d72ca'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:41.262273 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-22faf3f3-f186-4a41-bab3-7d93f780c0f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28c79a89b0c44cd396ce572fe03d72ca'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:41.275975 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-22faf3f3-f186-4a41-bab3-7d93f780c0f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28c79a89b0c44cd396ce572fe03d72ca'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:41.373488 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1282/2564] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:31:40 2022] DELETE /identity/v3/users/28c79a89b0c44cd396ce572fe03d72ca => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:41.378190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b9ccc-7b33-40fa-8489-1c4b67c09da8 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:41.378558 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b9ccc-7b33-40fa-8489-1c4b67c09da8 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:41.378898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b9ccc-7b33-40fa-8489-1c4b67c09da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:41.409157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-ce6b9ccc-7b33-40fa-8489-1c4b67c09da8 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:41.410602 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ce6b9ccc-7b33-40fa-8489-1c4b67c09da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:41.454703 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:41.481413 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1283/2565] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:41.495983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22faf3f3-f186-4a41-bab3-7d93f780c0f2 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:41.498186 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-22faf3f3-f186-4a41-bab3-7d93f780c0f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:41.566455 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22faf3f3-f186-4a41-bab3-7d93f780c0f2 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:41.571141 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715a576-5e77-4259-b362-f54e41309849 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:41.571903 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715a576-5e77-4259-b362-f54e41309849 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:41.572193 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715a576-5e77-4259-b362-f54e41309849 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:41.572472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715a576-5e77-4259-b362-f54e41309849 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:41.584511 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715a576-5e77-4259-b362-f54e41309849 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:41.586022 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715a576-5e77-4259-b362-f54e41309849 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:41.598263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1283/2566] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:31:42.176677 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b9ccc-7b33-40fa-8489-1c4b67c09da8 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:42.177141 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ce6b9ccc-7b33-40fa-8489-1c4b67c09da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:42.189450 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b9ccc-7b33-40fa-8489-1c4b67c09da8 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:42.194703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa5f9a-08a7-4d6e-bd72-d1dd73c01a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:42.196070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa5f9a-08a7-4d6e-bd72-d1dd73c01a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:42.196956 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa5f9a-08a7-4d6e-bd72-d1dd73c01a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:42.198230 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa5f9a-08a7-4d6e-bd72-d1dd73c01a15 admin admin] PATH_INFO: `/v3/projects/d3e481e5158b499c8da9dd6fb2da2a9f` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:42.200344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:42.207860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa5f9a-08a7-4d6e-bd72-d1dd73c01a15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3e481e5158b499c8da9dd6fb2da2a9f)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:42.211751 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa5f9a-08a7-4d6e-bd72-d1dd73c01a15 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:42.381123 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-88aa5f9a-08a7-4d6e-bd72-d1dd73c01a15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3e481e5158b499c8da9dd6fb2da2a9f', 'request_id': 'req-88aa5f9a-08a7-4d6e-bd72-d1dd73c01a15'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:42.387887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-88aa5f9a-08a7-4d6e-bd72-d1dd73c01a15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3e481e5158b499c8da9dd6fb2da2a9f', 'request_id': 'req-88aa5f9a-08a7-4d6e-bd72-d1dd73c01a15'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:42.545361 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-88aa5f9a-08a7-4d6e-bd72-d1dd73c01a15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3e481e5158b499c8da9dd6fb2da2a9f', 'request_id': 'req-88aa5f9a-08a7-4d6e-bd72-d1dd73c01a15'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:42.557654 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-88aa5f9a-08a7-4d6e-bd72-d1dd73c01a15 admin admin] The token cache is being invalidate because project d3e481e5158b499c8da9dd6fb2da2a9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:31:42.558021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-88aa5f9a-08a7-4d6e-bd72-d1dd73c01a15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:42.558664 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-88aa5f9a-08a7-4d6e-bd72-d1dd73c01a15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:42.560544 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1284/2567] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:31:42 2022] DELETE /identity/v3/projects/d3e481e5158b499c8da9dd6fb2da2a9f => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:42.564574 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715a576-5e77-4259-b362-f54e41309849 service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:42.574320 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715a576-5e77-4259-b362-f54e41309849 service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:42.637219 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76678da2-5cc3-4d35-a1eb-4e98cc9d54fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:42.638300 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76678da2-5cc3-4d35-a1eb-4e98cc9d54fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:42.638665 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76678da2-5cc3-4d35-a1eb-4e98cc9d54fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:42.639058 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76678da2-5cc3-4d35-a1eb-4e98cc9d54fd admin admin] PATH_INFO: `/v3/users/709f8e6f6d154ad4990122378f5d3dda` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:42.640558 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76678da2-5cc3-4d35-a1eb-4e98cc9d54fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=709f8e6f6d154ad4990122378f5d3dda)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:42.643067 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76678da2-5cc3-4d35-a1eb-4e98cc9d54fd admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:42.681893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa5f9a-08a7-4d6e-bd72-d1dd73c01a15 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:42.683401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-88aa5f9a-08a7-4d6e-bd72-d1dd73c01a15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:42.697199 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa5f9a-08a7-4d6e-bd72-d1dd73c01a15 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:42.701888 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c057a-20dc-4ad8-899f-fc0df5a28ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:42.702813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c057a-20dc-4ad8-899f-fc0df5a28ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:42.703133 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c057a-20dc-4ad8-899f-fc0df5a28ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:42.703420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c057a-20dc-4ad8-899f-fc0df5a28ea3 admin admin] PATH_INFO: `/v3/users/69a7493862484a078dd9b33d97068149` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:42.704633 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7c057a-20dc-4ad8-899f-fc0df5a28ea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69a7493862484a078dd9b33d97068149)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:42.706577 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7c057a-20dc-4ad8-899f-fc0df5a28ea3 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:42.905590 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-76678da2-5cc3-4d35-a1eb-4e98cc9d54fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '709f8e6f6d154ad4990122378f5d3dda'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:42.923482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-76678da2-5cc3-4d35-a1eb-4e98cc9d54fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '709f8e6f6d154ad4990122378f5d3dda'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:42.924222 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-76678da2-5cc3-4d35-a1eb-4e98cc9d54fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '709f8e6f6d154ad4990122378f5d3dda'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:42.924845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-76678da2-5cc3-4d35-a1eb-4e98cc9d54fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '709f8e6f6d154ad4990122378f5d3dda'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:42.937938 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-76678da2-5cc3-4d35-a1eb-4e98cc9d54fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '709f8e6f6d154ad4990122378f5d3dda'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:42.999297 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4a7c057a-20dc-4ad8-899f-fc0df5a28ea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69a7493862484a078dd9b33d97068149'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:43.006647 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4a7c057a-20dc-4ad8-899f-fc0df5a28ea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69a7493862484a078dd9b33d97068149'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:43.007369 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4a7c057a-20dc-4ad8-899f-fc0df5a28ea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69a7493862484a078dd9b33d97068149'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:43.008539 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4a7c057a-20dc-4ad8-899f-fc0df5a28ea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69a7493862484a078dd9b33d97068149'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:43.022202 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4a7c057a-20dc-4ad8-899f-fc0df5a28ea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69a7493862484a078dd9b33d97068149'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:43.036300 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1284/2568] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:31:42 2022] DELETE /identity/v3/users/709f8e6f6d154ad4990122378f5d3dda => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:43.145116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1285/2569] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:31:42 2022] DELETE /identity/v3/users/69a7493862484a078dd9b33d97068149 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:31:43.362480 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:43.432902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76678da2-5cc3-4d35-a1eb-4e98cc9d54fd admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:43.435536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-76678da2-5cc3-4d35-a1eb-4e98cc9d54fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:43.507009 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76678da2-5cc3-4d35-a1eb-4e98cc9d54fd admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:43.524506 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c057a-20dc-4ad8-899f-fc0df5a28ea3 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:43.526146 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-4a7c057a-20dc-4ad8-899f-fc0df5a28ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:43.580335 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd749e-cd84-47b4-b5b1-858c6f0a8586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:43.582796 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd749e-cd84-47b4-b5b1-858c6f0a8586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:43.582796 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd749e-cd84-47b4-b5b1-858c6f0a8586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:43.582796 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd749e-cd84-47b4-b5b1-858c6f0a8586 admin admin] PATH_INFO: `/v3/projects/da9c33b4686f4655927d30cc24950141` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:43.583263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd749e-cd84-47b4-b5b1-858c6f0a8586 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da9c33b4686f4655927d30cc24950141)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:43.585106 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd749e-cd84-47b4-b5b1-858c6f0a8586 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:43.643386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c057a-20dc-4ad8-899f-fc0df5a28ea3 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:43.648031 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787488be-55ed-41b9-bb72-ea60ad61ba1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:43.648961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787488be-55ed-41b9-bb72-ea60ad61ba1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:43.649252 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787488be-55ed-41b9-bb72-ea60ad61ba1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:43.649533 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787488be-55ed-41b9-bb72-ea60ad61ba1c admin admin] PATH_INFO: `/v3/projects/73660378e226452db04380659e53d969` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:43.650728 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787488be-55ed-41b9-bb72-ea60ad61ba1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=73660378e226452db04380659e53d969)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:43.652705 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787488be-55ed-41b9-bb72-ea60ad61ba1c admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:43.707035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-cecd749e-cd84-47b4-b5b1-858c6f0a8586 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da9c33b4686f4655927d30cc24950141', 'request_id': 'req-cecd749e-cd84-47b4-b5b1-858c6f0a8586'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:43.713195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-cecd749e-cd84-47b4-b5b1-858c6f0a8586 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da9c33b4686f4655927d30cc24950141', 'request_id': 'req-cecd749e-cd84-47b4-b5b1-858c6f0a8586'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:43.718827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-cecd749e-cd84-47b4-b5b1-858c6f0a8586 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da9c33b4686f4655927d30cc24950141', 'request_id': 'req-cecd749e-cd84-47b4-b5b1-858c6f0a8586'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:43.780002 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-787488be-55ed-41b9-bb72-ea60ad61ba1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73660378e226452db04380659e53d969', 'request_id': 'req-787488be-55ed-41b9-bb72-ea60ad61ba1c'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:43.786098 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-787488be-55ed-41b9-bb72-ea60ad61ba1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73660378e226452db04380659e53d969', 'request_id': 'req-787488be-55ed-41b9-bb72-ea60ad61ba1c'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:43.791414 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:43.864775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-cecd749e-cd84-47b4-b5b1-858c6f0a8586 admin admin] The token cache is being invalidate because project da9c33b4686f4655927d30cc24950141 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:31:43.865076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-cecd749e-cd84-47b4-b5b1-858c6f0a8586 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:43.865699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-cecd749e-cd84-47b4-b5b1-858c6f0a8586 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:43.867441 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1285/2570] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:31:43 2022] DELETE /identity/v3/projects/da9c33b4686f4655927d30cc24950141 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:43.871373 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd749e-cd84-47b4-b5b1-858c6f0a8586 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:43.872681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-cecd749e-cd84-47b4-b5b1-858c6f0a8586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:43.938728 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd749e-cd84-47b4-b5b1-858c6f0a8586 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:43.938998 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-787488be-55ed-41b9-bb72-ea60ad61ba1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73660378e226452db04380659e53d969', 'request_id': 'req-787488be-55ed-41b9-bb72-ea60ad61ba1c'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:43.950971 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-787488be-55ed-41b9-bb72-ea60ad61ba1c admin admin] The token cache is being invalidate because project 73660378e226452db04380659e53d969 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:31:43.951313 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-787488be-55ed-41b9-bb72-ea60ad61ba1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:43.951964 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-787488be-55ed-41b9-bb72-ea60ad61ba1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:43.953595 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1286/2571] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:31:43 2022] DELETE /identity/v3/projects/73660378e226452db04380659e53d969 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:31:43.959112 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65521f02-ae6a-4c17-8e2a-83fb49cf38eb None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:43.959525 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65521f02-ae6a-4c17-8e2a-83fb49cf38eb None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:43.960052 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65521f02-ae6a-4c17-8e2a-83fb49cf38eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:43.980517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-65521f02-ae6a-4c17-8e2a-83fb49cf38eb None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:43.981794 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-65521f02-ae6a-4c17-8e2a-83fb49cf38eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:44.001401 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b0399-2958-4c47-a1fa-77ed67f062c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:44.002290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b0399-2958-4c47-a1fa-77ed67f062c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:44.002581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b0399-2958-4c47-a1fa-77ed67f062c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:44.002890 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b0399-2958-4c47-a1fa-77ed67f062c7 admin admin] PATH_INFO: `/v3/users/b20a52f21c3c4c5d9ab7019d1965d73f` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:44.004142 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b0399-2958-4c47-a1fa-77ed67f062c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b20a52f21c3c4c5d9ab7019d1965d73f)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:44.006091 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b0399-2958-4c47-a1fa-77ed67f062c7 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:44.049491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1287/2572] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:31:44.058536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfec481-e7b1-44b7-8ed1-124cf4d25154 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:44.058830 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfec481-e7b1-44b7-8ed1-124cf4d25154 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:44.059173 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfec481-e7b1-44b7-8ed1-124cf4d25154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:44.078208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-5dfec481-e7b1-44b7-8ed1-124cf4d25154 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:44.079494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5dfec481-e7b1-44b7-8ed1-124cf4d25154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:44.088289 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1288/2573] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:31:44.092450 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfec481-e7b1-44b7-8ed1-124cf4d25154 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:44.101928 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfec481-e7b1-44b7-8ed1-124cf4d25154 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:44.106662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec58a1-43a6-4399-aad6-d297c1927896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:44.107984 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec58a1-43a6-4399-aad6-d297c1927896 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:44.107984 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec58a1-43a6-4399-aad6-d297c1927896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:44.107984 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec58a1-43a6-4399-aad6-d297c1927896 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:44.108632 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec58a1-43a6-4399-aad6-d297c1927896 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:31:44.108949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec58a1-43a6-4399-aad6-d297c1927896 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:44.110567 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec58a1-43a6-4399-aad6-d297c1927896 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:44.122376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1289/2574] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:31:44.127267 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec58a1-43a6-4399-aad6-d297c1927896 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:44.139304 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec58a1-43a6-4399-aad6-d297c1927896 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:44.144098 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86461f-ee16-470c-a81a-5906d27562d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:44.144765 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86461f-ee16-470c-a81a-5906d27562d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:44.145055 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86461f-ee16-470c-a81a-5906d27562d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:44.145334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86461f-ee16-470c-a81a-5906d27562d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:44.146032 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd86461f-ee16-470c-a81a-5906d27562d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:44.148002 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd86461f-ee16-470c-a81a-5906d27562d1 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:44.195004 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-f16b0399-2958-4c47-a1fa-77ed67f062c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b20a52f21c3c4c5d9ab7019d1965d73f'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:44.201982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-f16b0399-2958-4c47-a1fa-77ed67f062c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b20a52f21c3c4c5d9ab7019d1965d73f'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:44.202686 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-f16b0399-2958-4c47-a1fa-77ed67f062c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b20a52f21c3c4c5d9ab7019d1965d73f'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:44.203333 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-f16b0399-2958-4c47-a1fa-77ed67f062c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b20a52f21c3c4c5d9ab7019d1965d73f'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:44.215502 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-f16b0399-2958-4c47-a1fa-77ed67f062c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b20a52f21c3c4c5d9ab7019d1965d73f'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:44.262002 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1290/2575] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:31:44 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:31:44.266316 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86461f-ee16-470c-a81a-5906d27562d1 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:44.268005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-fd86461f-ee16-470c-a81a-5906d27562d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:44.271985 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:44.278344 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1286/2576] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:31:43 2022] DELETE /identity/v3/users/b20a52f21c3c4c5d9ab7019d1965d73f => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:31:44.334028 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86461f-ee16-470c-a81a-5906d27562d1 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:44.391929 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef9856-106c-407b-af77-6c902a97ed56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:44.392637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef9856-106c-407b-af77-6c902a97ed56 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:44.392921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef9856-106c-407b-af77-6c902a97ed56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:44.393197 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef9856-106c-407b-af77-6c902a97ed56 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:44.393888 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef9856-106c-407b-af77-6c902a97ed56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:44.397852 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef9856-106c-407b-af77-6c902a97ed56 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:44.514720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1291/2577] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:31:44 2022] POST /identity/v3/users => generated 293 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:31:44.516694 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b0399-2958-4c47-a1fa-77ed67f062c7 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:44.527480 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b0399-2958-4c47-a1fa-77ed67f062c7 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:44.532733 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086acd7-9917-4e46-98c6-cc4890c2b003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:44.533581 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086acd7-9917-4e46-98c6-cc4890c2b003 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:44.533869 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086acd7-9917-4e46-98c6-cc4890c2b003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:44.534151 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086acd7-9917-4e46-98c6-cc4890c2b003 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:44.534888 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086acd7-9917-4e46-98c6-cc4890c2b003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:44.536530 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086acd7-9917-4e46-98c6-cc4890c2b003 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:44.540138 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:44.547279 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef9856-106c-407b-af77-6c902a97ed56 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:44.549325 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-10ef9856-106c-407b-af77-6c902a97ed56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:44.549799 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1287/2578] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:31:44.553371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086acd7-9917-4e46-98c6-cc4890c2b003 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:44.564501 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086acd7-9917-4e46-98c6-cc4890c2b003 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:44.570088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e8051-fa26-4a04-8fa1-b69c8ea76a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:44.571629 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e8051-fa26-4a04-8fa1-b69c8ea76a00 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:44.571924 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e8051-fa26-4a04-8fa1-b69c8ea76a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:44.572201 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e8051-fa26-4a04-8fa1-b69c8ea76a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:44.573130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e8051-fa26-4a04-8fa1-b69c8ea76a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:44.574669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef9856-106c-407b-af77-6c902a97ed56 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:44.574970 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e8051-fa26-4a04-8fa1-b69c8ea76a00 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:44.580324 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a302a-7a7a-44a2-b97a-8ba07f24b623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:44.581193 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a302a-7a7a-44a2-b97a-8ba07f24b623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:44.581478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a302a-7a7a-44a2-b97a-8ba07f24b623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:44.581933 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a302a-7a7a-44a2-b97a-8ba07f24b623 admin admin] PATH_INFO: `/v3/projects/13798ae7abba44a8893cd5401d1684f0` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:44.583184 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a302a-7a7a-44a2-b97a-8ba07f24b623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13798ae7abba44a8893cd5401d1684f0)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:44.585705 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a302a-7a7a-44a2-b97a-8ba07f24b623 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:44.591350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1288/2579] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:31:44.591350 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:44.591961 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e8051-fa26-4a04-8fa1-b69c8ea76a00 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:44.602761 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e8051-fa26-4a04-8fa1-b69c8ea76a00 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:44.608397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11889c-f316-42c8-82d9-2b88bb8e23f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:44.609302 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11889c-f316-42c8-82d9-2b88bb8e23f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:44.609593 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11889c-f316-42c8-82d9-2b88bb8e23f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:44.609877 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11889c-f316-42c8-82d9-2b88bb8e23f4 admin admin] PATH_INFO: `/v3/projects/8129a8935079469cb87dd09dd1c8ef36/users/f8240033a5224ced8a69435748da1a2e/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:44.627092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11889c-f316-42c8-82d9-2b88bb8e23f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8129a8935079469cb87dd09dd1c8ef36, user_id=f8240033a5224ced8a69435748da1a2e, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:44.629827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11889c-f316-42c8-82d9-2b88bb8e23f4 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:44.757097 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1289/2580] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:44 2022] PUT /identity/v3/projects/8129a8935079469cb87dd09dd1c8ef36/users/f8240033a5224ced8a69435748da1a2e/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:44.766862 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2b1a302a-7a7a-44a2-b97a-8ba07f24b623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13798ae7abba44a8893cd5401d1684f0', 'request_id': 'req-2b1a302a-7a7a-44a2-b97a-8ba07f24b623'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:44.774426 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2b1a302a-7a7a-44a2-b97a-8ba07f24b623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13798ae7abba44a8893cd5401d1684f0', 'request_id': 'req-2b1a302a-7a7a-44a2-b97a-8ba07f24b623'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:44.776398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f93e41-662d-4f91-9a80-e53289d03eb2 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:44.776696 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f93e41-662d-4f91-9a80-e53289d03eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:44.776983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f93e41-662d-4f91-9a80-e53289d03eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:44.803645 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-47f93e41-662d-4f91-9a80-e53289d03eb2 None None] MFA Rules not processed for user `f8240033a5224ced8a69435748da1a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:44.804882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-47f93e41-662d-4f91-9a80-e53289d03eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:44.859980 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2b1a302a-7a7a-44a2-b97a-8ba07f24b623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13798ae7abba44a8893cd5401d1684f0', 'request_id': 'req-2b1a302a-7a7a-44a2-b97a-8ba07f24b623'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:44.871639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2b1a302a-7a7a-44a2-b97a-8ba07f24b623 admin admin] The token cache is being invalidate because project 13798ae7abba44a8893cd5401d1684f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:31:44.871991 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2b1a302a-7a7a-44a2-b97a-8ba07f24b623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:44.872633 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2b1a302a-7a7a-44a2-b97a-8ba07f24b623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:44.874196 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1292/2581] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:31:44 2022] DELETE /identity/v3/projects/13798ae7abba44a8893cd5401d1684f0 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:31:44.875595 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1290/2582] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:31:44.878882 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94506a-081a-4e35-88d9-bff2b733f3f8 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:44.879266 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94506a-081a-4e35-88d9-bff2b733f3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:44.879610 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f93e41-662d-4f91-9a80-e53289d03eb2 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:44.879732 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94506a-081a-4e35-88d9-bff2b733f3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:44.900731 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-fa94506a-081a-4e35-88d9-bff2b733f3f8 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:44.904701 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-fa94506a-081a-4e35-88d9-bff2b733f3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:44.944787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f93e41-662d-4f91-9a80-e53289d03eb2 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:44.951437 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173d57c-de04-4155-aaff-aa74abd123ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:44.952392 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173d57c-de04-4155-aaff-aa74abd123ff admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:44.952684 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173d57c-de04-4155-aaff-aa74abd123ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:44.952959 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173d57c-de04-4155-aaff-aa74abd123ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:44.953706 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173d57c-de04-4155-aaff-aa74abd123ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:44.955637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173d57c-de04-4155-aaff-aa74abd123ff admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:44.965285 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1293/2583] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:44.973382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073a145-4ba0-4bab-80b9-8f900e742196 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:44.973679 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073a145-4ba0-4bab-80b9-8f900e742196 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:44.974049 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073a145-4ba0-4bab-80b9-8f900e742196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:44.994401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-2073a145-4ba0-4bab-80b9-8f900e742196 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:44.995709 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-2073a145-4ba0-4bab-80b9-8f900e742196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:45.004679 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1294/2584] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:31:45.009080 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073a145-4ba0-4bab-80b9-8f900e742196 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:45.019522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073a145-4ba0-4bab-80b9-8f900e742196 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:45.026562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83481641-d08b-427d-b0b4-b5885f343fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:45.027268 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83481641-d08b-427d-b0b4-b5885f343fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:45.027605 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83481641-d08b-427d-b0b4-b5885f343fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:45.027847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83481641-d08b-427d-b0b4-b5885f343fb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:45.028570 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83481641-d08b-427d-b0b4-b5885f343fb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:31:45.028877 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83481641-d08b-427d-b0b4-b5885f343fb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:45.030488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83481641-d08b-427d-b0b4-b5885f343fb7 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:45.043953 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1295/2585] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:31:45.049034 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83481641-d08b-427d-b0b4-b5885f343fb7 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:45.059953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1291/2586] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:31:44 2022] POST /identity/v3/projects => generated 385 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:31:45.063599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173d57c-de04-4155-aaff-aa74abd123ff admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:45.065154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83481641-d08b-427d-b0b4-b5885f343fb7 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:45.065792 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-9173d57c-de04-4155-aaff-aa74abd123ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:45.134471 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173d57c-de04-4155-aaff-aa74abd123ff admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:45.142054 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22c18c-7cda-462f-9afc-820a8092e267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:45.142054 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22c18c-7cda-462f-9afc-820a8092e267 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:45.142054 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22c18c-7cda-462f-9afc-820a8092e267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:45.142054 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22c18c-7cda-462f-9afc-820a8092e267 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:45.142436 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22c18c-7cda-462f-9afc-820a8092e267 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:45.142539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e700125-accd-4f5f-bb81-bab381a6461a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:45.142539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e700125-accd-4f5f-bb81-bab381a6461a admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:45.142539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e700125-accd-4f5f-bb81-bab381a6461a admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:45.142539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e700125-accd-4f5f-bb81-bab381a6461a admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:45.142888 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22c18c-7cda-462f-9afc-820a8092e267 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:45.143045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e700125-accd-4f5f-bb81-bab381a6461a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:45.143045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e700125-accd-4f5f-bb81-bab381a6461a admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:45.299870 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1292/2587] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:31:45 2022] POST /identity/v3/users => generated 299 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:31:45.305067 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e700125-accd-4f5f-bb81-bab381a6461a admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:45.316620 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e700125-accd-4f5f-bb81-bab381a6461a admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:45.322211 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc9fc5-4a48-403d-bd7f-81937096bcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:45.322889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc9fc5-4a48-403d-bd7f-81937096bcac admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:45.323215 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc9fc5-4a48-403d-bd7f-81937096bcac admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:45.327264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc9fc5-4a48-403d-bd7f-81937096bcac admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:45.327394 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1296/2588] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:31:45 2022] POST /identity/v3/projects => generated 363 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:31:45.327533 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc9fc5-4a48-403d-bd7f-81937096bcac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:45.327533 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc9fc5-4a48-403d-bd7f-81937096bcac admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:45.330091 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22c18c-7cda-462f-9afc-820a8092e267 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:45.340215 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1293/2589] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:45.344230 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc9fc5-4a48-403d-bd7f-81937096bcac admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:45.354570 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc9fc5-4a48-403d-bd7f-81937096bcac admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:45.397402 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22c18c-7cda-462f-9afc-820a8092e267 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:45.402975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdb379-6d8c-43c4-9fe6-b6342e3cbd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:45.403669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdb379-6d8c-43c4-9fe6-b6342e3cbd76 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:45.403959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdb379-6d8c-43c4-9fe6-b6342e3cbd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:45.404240 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdb379-6d8c-43c4-9fe6-b6342e3cbd76 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:45.404940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebdb379-6d8c-43c4-9fe6-b6342e3cbd76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:45.407013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebdb379-6d8c-43c4-9fe6-b6342e3cbd76 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:45.412743 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96813f-39a4-4d74-b1b5-0e6b76fd652b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:45.413708 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96813f-39a4-4d74-b1b5-0e6b76fd652b admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:45.414526 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96813f-39a4-4d74-b1b5-0e6b76fd652b admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:45.414809 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96813f-39a4-4d74-b1b5-0e6b76fd652b admin admin] PATH_INFO: `/v3/projects/8e75370aebfc44ea8a3cb6c7f7fc592c/users/721ec473b8e842389a8863e4d3fa7dbb/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:45.429604 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96813f-39a4-4d74-b1b5-0e6b76fd652b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e75370aebfc44ea8a3cb6c7f7fc592c, user_id=721ec473b8e842389a8863e4d3fa7dbb, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:45.432906 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96813f-39a4-4d74-b1b5-0e6b76fd652b admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:45.482334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1297/2590] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:31:45 2022] POST /identity/v3/users => generated 282 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:31:45.486282 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdb379-6d8c-43c4-9fe6-b6342e3cbd76 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:45.496079 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdb379-6d8c-43c4-9fe6-b6342e3cbd76 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:45.501102 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f3156-4127-49df-8f26-6423323c20f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:45.501774 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f3156-4127-49df-8f26-6423323c20f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:45.502060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f3156-4127-49df-8f26-6423323c20f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:45.502337 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f3156-4127-49df-8f26-6423323c20f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:45.503144 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f3156-4127-49df-8f26-6423323c20f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:45.508194 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f3156-4127-49df-8f26-6423323c20f5 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:45.508334 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1294/2591] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:45 2022] PUT /identity/v3/projects/8e75370aebfc44ea8a3cb6c7f7fc592c/users/721ec473b8e842389a8863e4d3fa7dbb/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:45.510788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96813f-39a4-4d74-b1b5-0e6b76fd652b admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:45.515951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1298/2592] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:31:45.519812 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f3156-4127-49df-8f26-6423323c20f5 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:45.530813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f3156-4127-49df-8f26-6423323c20f5 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:45.575842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96813f-39a4-4d74-b1b5-0e6b76fd652b admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:45.580848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42887dc5-7367-492e-a9ad-bb1777ec708d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:45.581556 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42887dc5-7367-492e-a9ad-bb1777ec708d admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:45.581996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42887dc5-7367-492e-a9ad-bb1777ec708d admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:45.582160 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42887dc5-7367-492e-a9ad-bb1777ec708d admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:45.582901 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42887dc5-7367-492e-a9ad-bb1777ec708d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:45.584595 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42887dc5-7367-492e-a9ad-bb1777ec708d admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:45.592123 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b07a9-75ff-49f1-8303-d19f310f2cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:45.592804 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b07a9-75ff-49f1-8303-d19f310f2cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:45.593117 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b07a9-75ff-49f1-8303-d19f310f2cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:45.593392 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b07a9-75ff-49f1-8303-d19f310f2cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:45.594185 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b07a9-75ff-49f1-8303-d19f310f2cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:45.595860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b07a9-75ff-49f1-8303-d19f310f2cc3 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:45.596362 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1295/2593] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:31:45.600040 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42887dc5-7367-492e-a9ad-bb1777ec708d admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:45.606623 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1299/2594] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:31:45.611616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42887dc5-7367-492e-a9ad-bb1777ec708d admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:45.618421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b07a9-75ff-49f1-8303-d19f310f2cc3 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:45.620005 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2c3f4-3508-4be8-ab8c-97cb999c03ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:45.620919 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2c3f4-3508-4be8-ab8c-97cb999c03ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:45.621211 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2c3f4-3508-4be8-ab8c-97cb999c03ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:45.621494 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2c3f4-3508-4be8-ab8c-97cb999c03ed admin admin] PATH_INFO: `/v3/projects/8e75370aebfc44ea8a3cb6c7f7fc592c/users/721ec473b8e842389a8863e4d3fa7dbb/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:45.623591 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf2c3f4-3508-4be8-ab8c-97cb999c03ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e75370aebfc44ea8a3cb6c7f7fc592c, user_id=721ec473b8e842389a8863e4d3fa7dbb, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:45.626323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf2c3f4-3508-4be8-ab8c-97cb999c03ed admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:45.628528 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b07a9-75ff-49f1-8303-d19f310f2cc3 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:45.633764 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d5e27-03ac-4c30-ab43-7282ef4b57b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:45.634675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d5e27-03ac-4c30-ab43-7282ef4b57b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:45.634996 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d5e27-03ac-4c30-ab43-7282ef4b57b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:45.635280 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d5e27-03ac-4c30-ab43-7282ef4b57b9 admin admin] PATH_INFO: `/v3/projects/8f1a568e94c947c9bcd1f121bb11bc63/users/fd87140d99ec41d6afe861fc5325832f/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:45.637420 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1296/2595] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:45 2022] PUT /identity/v3/projects/8e75370aebfc44ea8a3cb6c7f7fc592c/users/721ec473b8e842389a8863e4d3fa7dbb/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:31:45.649723 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e3e19-ac02-4150-8cbd-225ae6d21565 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:45.649723 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e3e19-ac02-4150-8cbd-225ae6d21565 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:45.650020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e3e19-ac02-4150-8cbd-225ae6d21565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:45.652636 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d5e27-03ac-4c30-ab43-7282ef4b57b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f1a568e94c947c9bcd1f121bb11bc63, user_id=fd87140d99ec41d6afe861fc5325832f, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:45.655377 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d5e27-03ac-4c30-ab43-7282ef4b57b9 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:45.676484 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-b46e3e19-ac02-4150-8cbd-225ae6d21565 None None] MFA Rules not processed for user `721ec473b8e842389a8863e4d3fa7dbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:45.677707 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b46e3e19-ac02-4150-8cbd-225ae6d21565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:45.751015 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1297/2596] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:45 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:31:45.751898 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:45.759730 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1300/2597] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:45 2022] PUT /identity/v3/projects/8f1a568e94c947c9bcd1f121bb11bc63/users/fd87140d99ec41d6afe861fc5325832f/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:31:45.763566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e3e19-ac02-4150-8cbd-225ae6d21565 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:45.765085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b46e3e19-ac02-4150-8cbd-225ae6d21565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:45.772713 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cc708-94b2-400c-b469-1a8185db19fb None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:45.773022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cc708-94b2-400c-b469-1a8185db19fb None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:45.773297 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cc708-94b2-400c-b469-1a8185db19fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:45.801682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-585cc708-94b2-400c-b469-1a8185db19fb None None] MFA Rules not processed for user `fd87140d99ec41d6afe861fc5325832f`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:45.802961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-585cc708-94b2-400c-b469-1a8185db19fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:45.809660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:45.827459 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e3e19-ac02-4150-8cbd-225ae6d21565 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:45.871323 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1301/2598] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:45 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:31:45.875725 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cc708-94b2-400c-b469-1a8185db19fb None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:45.877368 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8844482-6aa1-4bfb-a1be-acadf3aedf89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:45.878024 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8844482-6aa1-4bfb-a1be-acadf3aedf89 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:45.878311 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8844482-6aa1-4bfb-a1be-acadf3aedf89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:45.878587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8844482-6aa1-4bfb-a1be-acadf3aedf89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:45.941695 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8844482-6aa1-4bfb-a1be-acadf3aedf89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:45.942635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cc708-94b2-400c-b469-1a8185db19fb None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:45.943233 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8844482-6aa1-4bfb-a1be-acadf3aedf89 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:45.947603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99034e3a-752f-4192-a75a-3c1253eeea29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:45.948269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99034e3a-752f-4192-a75a-3c1253eeea29 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:45.948558 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99034e3a-752f-4192-a75a-3c1253eeea29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:45.948835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99034e3a-752f-4192-a75a-3c1253eeea29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:45.949557 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99034e3a-752f-4192-a75a-3c1253eeea29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:45.952989 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99034e3a-752f-4192-a75a-3c1253eeea29 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:45.957496 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1298/2599] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:45 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:31:45.987044 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8844482-6aa1-4bfb-a1be-acadf3aedf89 service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:45.998371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8844482-6aa1-4bfb-a1be-acadf3aedf89 service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:46.003109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821df47-28fc-47ed-a222-7b5b845de685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:46.003988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821df47-28fc-47ed-a222-7b5b845de685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:46.004277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821df47-28fc-47ed-a222-7b5b845de685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:46.004556 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821df47-28fc-47ed-a222-7b5b845de685 admin admin] PATH_INFO: `/v3/users/f8240033a5224ced8a69435748da1a2e` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:46.005786 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821df47-28fc-47ed-a222-7b5b845de685 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8240033a5224ced8a69435748da1a2e)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:46.007781 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821df47-28fc-47ed-a222-7b5b845de685 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:46.038199 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1302/2600] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:31:45 2022] POST /identity/v3/projects => generated 365 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:31:46.042781 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99034e3a-752f-4192-a75a-3c1253eeea29 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:46.109209 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99034e3a-752f-4192-a75a-3c1253eeea29 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:46.114560 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2f3f4-d6a8-41d8-ba6d-e415b429344b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:46.115028 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2f3f4-d6a8-41d8-ba6d-e415b429344b admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:46.115289 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2f3f4-d6a8-41d8-ba6d-e415b429344b admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:46.115570 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2f3f4-d6a8-41d8-ba6d-e415b429344b admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:46.116272 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae2f3f4-d6a8-41d8-ba6d-e415b429344b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:46.118273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae2f3f4-d6a8-41d8-ba6d-e415b429344b admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:46.197502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1303/2601] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:31:46 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:31:46.201379 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2f3f4-d6a8-41d8-ba6d-e415b429344b admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:46.211805 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2f3f4-d6a8-41d8-ba6d-e415b429344b admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:46.216570 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907dc135-b9bb-4f57-8f89-486d41fcd24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:46.217247 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907dc135-b9bb-4f57-8f89-486d41fcd24e admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:46.217537 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907dc135-b9bb-4f57-8f89-486d41fcd24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:46.217820 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907dc135-b9bb-4f57-8f89-486d41fcd24e admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:46.218558 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907dc135-b9bb-4f57-8f89-486d41fcd24e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:46.220235 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907dc135-b9bb-4f57-8f89-486d41fcd24e admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:46.230747 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1304/2602] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:31:46.234962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907dc135-b9bb-4f57-8f89-486d41fcd24e admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:46.245096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907dc135-b9bb-4f57-8f89-486d41fcd24e admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:46.249907 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68725d31-a642-4d06-8c60-71381df7eaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:46.252219 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68725d31-a642-4d06-8c60-71381df7eaa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:46.252219 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68725d31-a642-4d06-8c60-71381df7eaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:46.252219 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68725d31-a642-4d06-8c60-71381df7eaa2 admin admin] PATH_INFO: `/v3/projects/ebef946ffbac49cfa8c2172ccfd75795/users/002ddead00fd47ba93b2b111bb1f90d3/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:46.269623 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68725d31-a642-4d06-8c60-71381df7eaa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebef946ffbac49cfa8c2172ccfd75795, user_id=002ddead00fd47ba93b2b111bb1f90d3, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:46.272353 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68725d31-a642-4d06-8c60-71381df7eaa2 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:46.284809 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-3821df47-28fc-47ed-a222-7b5b845de685 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8240033a5224ced8a69435748da1a2e'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:46.292101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-3821df47-28fc-47ed-a222-7b5b845de685 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8240033a5224ced8a69435748da1a2e'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:46.292843 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-3821df47-28fc-47ed-a222-7b5b845de685 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8240033a5224ced8a69435748da1a2e'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:46.293438 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-3821df47-28fc-47ed-a222-7b5b845de685 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8240033a5224ced8a69435748da1a2e'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:46.306213 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-3821df47-28fc-47ed-a222-7b5b845de685 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8240033a5224ced8a69435748da1a2e'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:46.386311 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1305/2603] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:46 2022] PUT /identity/v3/projects/ebef946ffbac49cfa8c2172ccfd75795/users/002ddead00fd47ba93b2b111bb1f90d3/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:31:46.390737 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68725d31-a642-4d06-8c60-71381df7eaa2 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:46.392731 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-68725d31-a642-4d06-8c60-71381df7eaa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:46.397092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1299/2604] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:31:45 2022] DELETE /identity/v3/users/f8240033a5224ced8a69435748da1a2e => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:31:46.402118 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232b638-61f6-4412-94cf-ba0408d17201 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:46.402488 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232b638-61f6-4412-94cf-ba0408d17201 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:46.402837 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232b638-61f6-4412-94cf-ba0408d17201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:46.421510 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-f232b638-61f6-4412-94cf-ba0408d17201 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:46.422845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-f232b638-61f6-4412-94cf-ba0408d17201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:46.475004 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:46.479808 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68725d31-a642-4d06-8c60-71381df7eaa2 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:46.484636 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edb666-b808-4fdd-b408-40b317406703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:46.485422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1300/2605] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:31:46.485548 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edb666-b808-4fdd-b408-40b317406703 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:46.485896 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edb666-b808-4fdd-b408-40b317406703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:46.486111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edb666-b808-4fdd-b408-40b317406703 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:46.486937 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51edb666-b808-4fdd-b408-40b317406703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:46.489325 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51edb666-b808-4fdd-b408-40b317406703 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:46.496896 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232b638-61f6-4412-94cf-ba0408d17201 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:46.498691 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-f232b638-61f6-4412-94cf-ba0408d17201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:46.500331 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:46.500757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1306/2606] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:46.504799 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edb666-b808-4fdd-b408-40b317406703 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:46.515901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edb666-b808-4fdd-b408-40b317406703 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:46.520954 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd3dbc-3500-4286-a56c-a54afb545bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:46.521876 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd3dbc-3500-4286-a56c-a54afb545bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:46.522165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd3dbc-3500-4286-a56c-a54afb545bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:46.522451 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd3dbc-3500-4286-a56c-a54afb545bbe admin admin] PATH_INFO: `/v3/projects/ebef946ffbac49cfa8c2172ccfd75795/users/002ddead00fd47ba93b2b111bb1f90d3/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:46.525891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd3dbc-3500-4286-a56c-a54afb545bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebef946ffbac49cfa8c2172ccfd75795, user_id=002ddead00fd47ba93b2b111bb1f90d3, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:46.528594 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd3dbc-3500-4286-a56c-a54afb545bbe admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:46.548297 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1307/2607] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:46 2022] PUT /identity/v3/projects/ebef946ffbac49cfa8c2172ccfd75795/users/002ddead00fd47ba93b2b111bb1f90d3/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:31:46.550064 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9effb2b-bec8-4700-93e8-a63e3e15c980 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:46.550459 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9effb2b-bec8-4700-93e8-a63e3e15c980 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:46.550816 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9effb2b-bec8-4700-93e8-a63e3e15c980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:46.568021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232b638-61f6-4412-94cf-ba0408d17201 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:46.580771 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-d9effb2b-bec8-4700-93e8-a63e3e15c980 None None] MFA Rules not processed for user `002ddead00fd47ba93b2b111bb1f90d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:46.583658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d9effb2b-bec8-4700-93e8-a63e3e15c980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:46.619976 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88921a9-85b2-47c5-a2db-28277c1d8cbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:46.620632 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88921a9-85b2-47c5-a2db-28277c1d8cbe service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:46.620927 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88921a9-85b2-47c5-a2db-28277c1d8cbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:46.621203 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88921a9-85b2-47c5-a2db-28277c1d8cbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:46.657191 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1308/2608] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:46 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:31:46.669116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9effb2b-bec8-4700-93e8-a63e3e15c980 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:46.670549 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d9effb2b-bec8-4700-93e8-a63e3e15c980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:46.690888 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88921a9-85b2-47c5-a2db-28277c1d8cbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:46.692446 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88921a9-85b2-47c5-a2db-28277c1d8cbe service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:46.703050 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1301/2609] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:31:46.738450 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9effb2b-bec8-4700-93e8-a63e3e15c980 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:46.744232 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f238862-95b8-4e3b-873c-0c7dd54e8136 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:46.744232 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f238862-95b8-4e3b-873c-0c7dd54e8136 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:46.744232 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f238862-95b8-4e3b-873c-0c7dd54e8136 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:46.744232 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f238862-95b8-4e3b-873c-0c7dd54e8136 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:46.754673 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f238862-95b8-4e3b-873c-0c7dd54e8136 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:46.767113 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f238862-95b8-4e3b-873c-0c7dd54e8136 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:46.768276 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1309/2610] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:46 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:31:47.084073 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88921a9-85b2-47c5-a2db-28277c1d8cbe service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:47.093894 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88921a9-85b2-47c5-a2db-28277c1d8cbe service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:47.097521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c936aa6-d6f8-4753-a98b-ea016cc179ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:47.098172 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c936aa6-d6f8-4753-a98b-ea016cc179ec service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:47.098454 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c936aa6-d6f8-4753-a98b-ea016cc179ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:47.098750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c936aa6-d6f8-4753-a98b-ea016cc179ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:47.100547 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-2c936aa6-d6f8-4753-a98b-ea016cc179ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:47.167978 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c936aa6-d6f8-4753-a98b-ea016cc179ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:47.169772 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c936aa6-d6f8-4753-a98b-ea016cc179ec service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:47.180662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1302/2611] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:47 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:31:47.216666 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f238862-95b8-4e3b-873c-0c7dd54e8136 service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:47.216666 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-5f238862-95b8-4e3b-873c-0c7dd54e8136 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:47.229165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f238862-95b8-4e3b-873c-0c7dd54e8136 service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:47.234361 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948d980-46d6-454d-a605-597846d9ad29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:47.235308 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948d980-46d6-454d-a605-597846d9ad29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:47.235663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948d980-46d6-454d-a605-597846d9ad29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:47.236027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948d980-46d6-454d-a605-597846d9ad29 admin admin] PATH_INFO: `/v3/projects/8129a8935079469cb87dd09dd1c8ef36` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:47.237183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948d980-46d6-454d-a605-597846d9ad29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8129a8935079469cb87dd09dd1c8ef36)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:47.239456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948d980-46d6-454d-a605-597846d9ad29 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:47.346008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:47.363867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7948d980-46d6-454d-a605-597846d9ad29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8129a8935079469cb87dd09dd1c8ef36', 'request_id': 'req-7948d980-46d6-454d-a605-597846d9ad29'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:47.370690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7948d980-46d6-454d-a605-597846d9ad29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8129a8935079469cb87dd09dd1c8ef36', 'request_id': 'req-7948d980-46d6-454d-a605-597846d9ad29'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:47.465505 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:47.510549 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7948d980-46d6-454d-a605-597846d9ad29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8129a8935079469cb87dd09dd1c8ef36', 'request_id': 'req-7948d980-46d6-454d-a605-597846d9ad29'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:47.521806 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7948d980-46d6-454d-a605-597846d9ad29 admin admin] The token cache is being invalidate because project 8129a8935079469cb87dd09dd1c8ef36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:31:47.522182 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7948d980-46d6-454d-a605-597846d9ad29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:47.523112 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7948d980-46d6-454d-a605-597846d9ad29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:47.525058 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1310/2612] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:31:47 2022] DELETE /identity/v3/projects/8129a8935079469cb87dd09dd1c8ef36 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:47.531801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c936aa6-d6f8-4753-a98b-ea016cc179ec service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:47.540460 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c936aa6-d6f8-4753-a98b-ea016cc179ec service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:47.598793 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3b9e2-cace-4334-91a7-754e4f6df2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:47.599702 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3b9e2-cace-4334-91a7-754e4f6df2c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:47.599988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3b9e2-cace-4334-91a7-754e4f6df2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:47.600264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3b9e2-cace-4334-91a7-754e4f6df2c1 admin admin] PATH_INFO: `/v3/users/721ec473b8e842389a8863e4d3fa7dbb` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:47.602038 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd3b9e2-cace-4334-91a7-754e4f6df2c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=721ec473b8e842389a8863e4d3fa7dbb)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:47.604096 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd3b9e2-cace-4334-91a7-754e4f6df2c1 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:47.833626 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-9dd3b9e2-cace-4334-91a7-754e4f6df2c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '721ec473b8e842389a8863e4d3fa7dbb'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:47.841200 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-9dd3b9e2-cace-4334-91a7-754e4f6df2c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '721ec473b8e842389a8863e4d3fa7dbb'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:47.841905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-9dd3b9e2-cace-4334-91a7-754e4f6df2c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '721ec473b8e842389a8863e4d3fa7dbb'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:47.842534 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-9dd3b9e2-cace-4334-91a7-754e4f6df2c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '721ec473b8e842389a8863e4d3fa7dbb'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:47.854421 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-9dd3b9e2-cace-4334-91a7-754e4f6df2c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '721ec473b8e842389a8863e4d3fa7dbb'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:47.980236 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1303/2613] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:31:47 2022] DELETE /identity/v3/users/721ec473b8e842389a8863e4d3fa7dbb => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:48.122635 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:48.454752 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948d980-46d6-454d-a605-597846d9ad29 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:48.456241 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-7948d980-46d6-454d-a605-597846d9ad29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:48.459658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:48.471935 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948d980-46d6-454d-a605-597846d9ad29 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:48.540006 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42046a76-48a4-48f9-b316-48bd49451fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:48.540893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42046a76-48a4-48f9-b316-48bd49451fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:48.541194 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42046a76-48a4-48f9-b316-48bd49451fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:48.541459 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42046a76-48a4-48f9-b316-48bd49451fd0 admin admin] PATH_INFO: `/v3/projects/8e75370aebfc44ea8a3cb6c7f7fc592c` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:48.543163 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42046a76-48a4-48f9-b316-48bd49451fd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e75370aebfc44ea8a3cb6c7f7fc592c)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:48.545070 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42046a76-48a4-48f9-b316-48bd49451fd0 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:48.655185 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:48.675090 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-42046a76-48a4-48f9-b316-48bd49451fd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e75370aebfc44ea8a3cb6c7f7fc592c', 'request_id': 'req-42046a76-48a4-48f9-b316-48bd49451fd0'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:48.682148 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-42046a76-48a4-48f9-b316-48bd49451fd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e75370aebfc44ea8a3cb6c7f7fc592c', 'request_id': 'req-42046a76-48a4-48f9-b316-48bd49451fd0'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:48.756741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-42046a76-48a4-48f9-b316-48bd49451fd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e75370aebfc44ea8a3cb6c7f7fc592c', 'request_id': 'req-42046a76-48a4-48f9-b316-48bd49451fd0'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:48.768304 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-42046a76-48a4-48f9-b316-48bd49451fd0 admin admin] The token cache is being invalidate because project 8e75370aebfc44ea8a3cb6c7f7fc592c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:31:48.768656 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-42046a76-48a4-48f9-b316-48bd49451fd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:48.769448 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-42046a76-48a4-48f9-b316-48bd49451fd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:48.770984 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1311/2614] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:31:48 2022] DELETE /identity/v3/projects/8e75370aebfc44ea8a3cb6c7f7fc592c => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:48.776611 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0ba5e-3a60-4d34-b02d-1a18a6d79407 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:48.776918 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0ba5e-3a60-4d34-b02d-1a18a6d79407 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:48.777202 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0ba5e-3a60-4d34-b02d-1a18a6d79407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:48.800292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-c1f0ba5e-3a60-4d34-b02d-1a18a6d79407 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:48.801509 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-c1f0ba5e-3a60-4d34-b02d-1a18a6d79407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:48.865053 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1304/2615] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:48.876055 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13ef9a-303b-4786-acde-5b5247bca795 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:48.876352 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13ef9a-303b-4786-acde-5b5247bca795 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:48.876636 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13ef9a-303b-4786-acde-5b5247bca795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:48.901385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-7f13ef9a-303b-4786-acde-5b5247bca795 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:48.902617 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-7f13ef9a-303b-4786-acde-5b5247bca795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:48.913474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1312/2616] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:48.923207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f0ba5e-3a60-4d34-b02d-1a18a6d79407 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:48.933557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f0ba5e-3a60-4d34-b02d-1a18a6d79407 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:48.938462 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d06bdf-2355-4864-a162-de3617dc1e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:48.939164 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d06bdf-2355-4864-a162-de3617dc1e02 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:48.939647 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d06bdf-2355-4864-a162-de3617dc1e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:48.941001 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:48.941451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d06bdf-2355-4864-a162-de3617dc1e02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:48.942240 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d06bdf-2355-4864-a162-de3617dc1e02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:31:48.942558 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d06bdf-2355-4864-a162-de3617dc1e02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:48.944231 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d06bdf-2355-4864-a162-de3617dc1e02 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:48.957111 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1305/2617] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:31:48.960841 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13ef9a-303b-4786-acde-5b5247bca795 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:48.970176 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13ef9a-303b-4786-acde-5b5247bca795 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:48.974853 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de648a1-0413-42c0-bc54-7d91ff525277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:48.975542 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de648a1-0413-42c0-bc54-7d91ff525277 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:48.975832 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de648a1-0413-42c0-bc54-7d91ff525277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:48.976115 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de648a1-0413-42c0-bc54-7d91ff525277 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:48.976859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de648a1-0413-42c0-bc54-7d91ff525277 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:48.978788 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de648a1-0413-42c0-bc54-7d91ff525277 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:49.124639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1313/2618] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:31:48 2022] POST /identity/v3/projects => generated 381 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:49.129041 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d06bdf-2355-4864-a162-de3617dc1e02 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:49.140136 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d06bdf-2355-4864-a162-de3617dc1e02 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:49.201266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338bb7df-f395-4b1c-bb46-397a45686295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:49.202069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bb7df-f395-4b1c-bb46-397a45686295 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:49.202400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bb7df-f395-4b1c-bb46-397a45686295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:49.202500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bb7df-f395-4b1c-bb46-397a45686295 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:49.203233 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338bb7df-f395-4b1c-bb46-397a45686295 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:49.205174 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338bb7df-f395-4b1c-bb46-397a45686295 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:49.311774 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1306/2619] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:31:49 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:49.322585 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de648a1-0413-42c0-bc54-7d91ff525277 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:49.332924 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de648a1-0413-42c0-bc54-7d91ff525277 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:49.337736 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c6d4b-a7c4-468a-93e6-78f512770bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:49.338474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c6d4b-a7c4-468a-93e6-78f512770bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:49.338829 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c6d4b-a7c4-468a-93e6-78f512770bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:49.339253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c6d4b-a7c4-468a-93e6-78f512770bf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:49.340115 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c6d4b-a7c4-468a-93e6-78f512770bf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:49.341857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c6d4b-a7c4-468a-93e6-78f512770bf4 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:49.352915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:49.359894 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1314/2620] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:49.363905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338bb7df-f395-4b1c-bb46-397a45686295 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:49.374943 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338bb7df-f395-4b1c-bb46-397a45686295 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:49.380150 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8178c1-9db5-436a-b425-ca0ea7ae9e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:49.380832 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8178c1-9db5-436a-b425-ca0ea7ae9e80 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:49.381126 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8178c1-9db5-436a-b425-ca0ea7ae9e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:49.381422 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8178c1-9db5-436a-b425-ca0ea7ae9e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:49.388293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8178c1-9db5-436a-b425-ca0ea7ae9e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:49.390021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8178c1-9db5-436a-b425-ca0ea7ae9e80 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:49.407530 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1307/2621] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:31:49.408498 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c6d4b-a7c4-468a-93e6-78f512770bf4 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:49.420049 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c6d4b-a7c4-468a-93e6-78f512770bf4 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:49.425250 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6855966-4fc1-47bf-b8e8-7a05a95d3e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:49.430334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6855966-4fc1-47bf-b8e8-7a05a95d3e98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:49.430334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6855966-4fc1-47bf-b8e8-7a05a95d3e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:49.430334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6855966-4fc1-47bf-b8e8-7a05a95d3e98 admin admin] PATH_INFO: `/v3/projects/96ddf3fb81d8436d81fa2d7026749369/users/57ad18ab796f494e848529f114245ea6/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:49.445671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6855966-4fc1-47bf-b8e8-7a05a95d3e98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96ddf3fb81d8436d81fa2d7026749369, user_id=57ad18ab796f494e848529f114245ea6, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:49.448502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6855966-4fc1-47bf-b8e8-7a05a95d3e98 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:49.557055 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1315/2622] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:49 2022] PUT /identity/v3/projects/96ddf3fb81d8436d81fa2d7026749369/users/57ad18ab796f494e848529f114245ea6/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:49.567440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2d7c0-672a-47c7-a3bf-e7d53e4fdd4b None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:49.567739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2d7c0-672a-47c7-a3bf-e7d53e4fdd4b None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:49.568255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2d7c0-672a-47c7-a3bf-e7d53e4fdd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:49.595712 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-60d2d7c0-672a-47c7-a3bf-e7d53e4fdd4b None None] MFA Rules not processed for user `57ad18ab796f494e848529f114245ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:49.596938 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-60d2d7c0-672a-47c7-a3bf-e7d53e4fdd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:49.668318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1308/2623] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:49.672867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6855966-4fc1-47bf-b8e8-7a05a95d3e98 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:49.744757 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6855966-4fc1-47bf-b8e8-7a05a95d3e98 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:49.750831 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973445ac-c26f-456c-b7a6-128c4be47624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:49.751561 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973445ac-c26f-456c-b7a6-128c4be47624 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:49.751874 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973445ac-c26f-456c-b7a6-128c4be47624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:49.752159 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973445ac-c26f-456c-b7a6-128c4be47624 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:49.753056 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973445ac-c26f-456c-b7a6-128c4be47624 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:49.755018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973445ac-c26f-456c-b7a6-128c4be47624 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:49.975222 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1316/2624] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:31:49 2022] POST /identity/v3/projects => generated 383 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:49.980164 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2d7c0-672a-47c7-a3bf-e7d53e4fdd4b None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:49.991256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2d7c0-672a-47c7-a3bf-e7d53e4fdd4b None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:50.048191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f3039-e1e3-40c4-874d-34c4cb137cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:50.048898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f3039-e1e3-40c4-874d-34c4cb137cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:50.049187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f3039-e1e3-40c4-874d-34c4cb137cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:50.049469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f3039-e1e3-40c4-874d-34c4cb137cb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:50.050253 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f3039-e1e3-40c4-874d-34c4cb137cb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:50.052482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f3039-e1e3-40c4-874d-34c4cb137cb6 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:50.218646 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1309/2625] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:31:49 2022] POST /identity/v3/users => generated 298 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:50.219872 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973445ac-c26f-456c-b7a6-128c4be47624 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:50.230480 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973445ac-c26f-456c-b7a6-128c4be47624 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:50.235001 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda1019-066d-42f1-b70c-bad90ef03dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:50.235657 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda1019-066d-42f1-b70c-bad90ef03dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:50.235947 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda1019-066d-42f1-b70c-bad90ef03dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:50.236305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda1019-066d-42f1-b70c-bad90ef03dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:50.237029 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda1019-066d-42f1-b70c-bad90ef03dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:50.238876 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda1019-066d-42f1-b70c-bad90ef03dd4 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:50.251360 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1317/2626] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:31:50.256844 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f3039-e1e3-40c4-874d-34c4cb137cb6 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:50.267539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f3039-e1e3-40c4-874d-34c4cb137cb6 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:50.272138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6bbe3-3929-4f4a-a300-19641950021a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:50.273014 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6bbe3-3929-4f4a-a300-19641950021a admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:50.273301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6bbe3-3929-4f4a-a300-19641950021a admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:50.273575 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6bbe3-3929-4f4a-a300-19641950021a admin admin] PATH_INFO: `/v3/projects/37249065f80a4097bbdad538cf58d59d/users/50f248be1d31469882376ed6f3b29555/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:50.288534 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6bbe3-3929-4f4a-a300-19641950021a admin admin] RBAC: Authorizing `identity:create_grant(project_id=37249065f80a4097bbdad538cf58d59d, user_id=50f248be1d31469882376ed6f3b29555, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:50.291236 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6bbe3-3929-4f4a-a300-19641950021a admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:50.305694 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:50.420259 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1310/2627] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:50 2022] PUT /identity/v3/projects/37249065f80a4097bbdad538cf58d59d/users/50f248be1d31469882376ed6f3b29555/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:50.425305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda1019-066d-42f1-b70c-bad90ef03dd4 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:50.437180 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda1019-066d-42f1-b70c-bad90ef03dd4 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:50.501385 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952843ef-557e-4827-aec9-478461a6be59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:50.502001 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952843ef-557e-4827-aec9-478461a6be59 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:50.502296 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952843ef-557e-4827-aec9-478461a6be59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:50.502579 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952843ef-557e-4827-aec9-478461a6be59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:50.503371 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952843ef-557e-4827-aec9-478461a6be59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:50.505016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952843ef-557e-4827-aec9-478461a6be59 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:50.515976 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1318/2628] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:50.520016 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6bbe3-3929-4f4a-a300-19641950021a admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:50.530321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6bbe3-3929-4f4a-a300-19641950021a admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:50.534976 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a2ea0-1c6b-4847-a608-3053c021c87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:50.535859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a2ea0-1c6b-4847-a608-3053c021c87c admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:50.536152 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a2ea0-1c6b-4847-a608-3053c021c87c admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:50.537407 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a2ea0-1c6b-4847-a608-3053c021c87c admin admin] PATH_INFO: `/v3/projects/37249065f80a4097bbdad538cf58d59d/users/50f248be1d31469882376ed6f3b29555/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:50.538536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a2ea0-1c6b-4847-a608-3053c021c87c admin admin] RBAC: Authorizing `identity:create_grant(project_id=37249065f80a4097bbdad538cf58d59d, user_id=50f248be1d31469882376ed6f3b29555, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:50.541232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a2ea0-1c6b-4847-a608-3053c021c87c admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:50.552287 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1311/2629] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:50 2022] PUT /identity/v3/projects/37249065f80a4097bbdad538cf58d59d/users/50f248be1d31469882376ed6f3b29555/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:50.563748 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fb860-3393-431f-b798-227762a6de5a None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:50.564270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fb860-3393-431f-b798-227762a6de5a None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:50.564551 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fb860-3393-431f-b798-227762a6de5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:50.594632 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-0c9fb860-3393-431f-b798-227762a6de5a None None] MFA Rules not processed for user `50f248be1d31469882376ed6f3b29555`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:50.595952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-0c9fb860-3393-431f-b798-227762a6de5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:50.669643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1319/2630] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:50.776727 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a2ea0-1c6b-4847-a608-3053c021c87c admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:50.784255 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-645a2ea0-1c6b-4847-a608-3053c021c87c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:50.839926 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a2ea0-1c6b-4847-a608-3053c021c87c admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:50.843498 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e63268-2af5-4343-8678-9961d53182d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:50.844155 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e63268-2af5-4343-8678-9961d53182d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:50.844440 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e63268-2af5-4343-8678-9961d53182d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:50.844720 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e63268-2af5-4343-8678-9961d53182d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:50.855054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e63268-2af5-4343-8678-9961d53182d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:50.856533 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e63268-2af5-4343-8678-9961d53182d2 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:50.870995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1312/2631] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:31:51.010189 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fb860-3393-431f-b798-227762a6de5a None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:51.019991 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fb860-3393-431f-b798-227762a6de5a None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:51.024014 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910927e-f493-4b69-b165-deb58a59a522 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:51.024709 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910927e-f493-4b69-b165-deb58a59a522 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:51.024999 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910927e-f493-4b69-b165-deb58a59a522 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:51.025290 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910927e-f493-4b69-b165-deb58a59a522 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:51.086901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910927e-f493-4b69-b165-deb58a59a522 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:51.088585 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910927e-f493-4b69-b165-deb58a59a522 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:51.098715 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1320/2632] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:31:51.102734 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e63268-2af5-4343-8678-9961d53182d2 service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:51.104263 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-56e63268-2af5-4343-8678-9961d53182d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:51.183756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e63268-2af5-4343-8678-9961d53182d2 service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:51.188518 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf05f7c-21fe-4a5f-a622-3691ee75d7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:51.189395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf05f7c-21fe-4a5f-a622-3691ee75d7cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:51.191712 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf05f7c-21fe-4a5f-a622-3691ee75d7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:51.191712 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf05f7c-21fe-4a5f-a622-3691ee75d7cc admin admin] PATH_INFO: `/v3/users/fd87140d99ec41d6afe861fc5325832f` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:51.191712 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf05f7c-21fe-4a5f-a622-3691ee75d7cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd87140d99ec41d6afe861fc5325832f)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:51.193085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf05f7c-21fe-4a5f-a622-3691ee75d7cc admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:51.339452 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:51.519613 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-3bf05f7c-21fe-4a5f-a622-3691ee75d7cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd87140d99ec41d6afe861fc5325832f'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:51.526878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-3bf05f7c-21fe-4a5f-a622-3691ee75d7cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd87140d99ec41d6afe861fc5325832f'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:51.527575 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-3bf05f7c-21fe-4a5f-a622-3691ee75d7cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd87140d99ec41d6afe861fc5325832f'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:51.528203 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-3bf05f7c-21fe-4a5f-a622-3691ee75d7cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd87140d99ec41d6afe861fc5325832f'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:51.540833 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-3bf05f7c-21fe-4a5f-a622-3691ee75d7cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd87140d99ec41d6afe861fc5325832f'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:51.781184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1313/2633] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:31:51 2022] DELETE /identity/v3/users/fd87140d99ec41d6afe861fc5325832f => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:31:51.786291 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f9ceb-ad7d-43d5-b105-f0b0325f3749 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:51.786585 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f9ceb-ad7d-43d5-b105-f0b0325f3749 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:51.786871 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f9ceb-ad7d-43d5-b105-f0b0325f3749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:51.805238 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-be2f9ceb-ad7d-43d5-b105-f0b0325f3749 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:51.806485 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-be2f9ceb-ad7d-43d5-b105-f0b0325f3749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:51.870344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1321/2634] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:31:51.879829 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf05f7c-21fe-4a5f-a622-3691ee75d7cc admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:51.881403 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-3bf05f7c-21fe-4a5f-a622-3691ee75d7cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:51.942401 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf05f7c-21fe-4a5f-a622-3691ee75d7cc admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:51.947104 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eed6cd-1081-4ea8-8733-4cc38569b03a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:51.947797 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eed6cd-1081-4ea8-8733-4cc38569b03a service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:51.948114 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eed6cd-1081-4ea8-8733-4cc38569b03a service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:51.948363 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eed6cd-1081-4ea8-8733-4cc38569b03a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:51.961632 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eed6cd-1081-4ea8-8733-4cc38569b03a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:51.963194 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eed6cd-1081-4ea8-8733-4cc38569b03a service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:51.972953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1314/2635] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:31:52.150975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f9ceb-ad7d-43d5-b105-f0b0325f3749 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:52.162333 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f9ceb-ad7d-43d5-b105-f0b0325f3749 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:52.167148 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d591ff7-743c-4172-bcc6-ee0b48cb221a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:52.172383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d591ff7-743c-4172-bcc6-ee0b48cb221a service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:52.172383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d591ff7-743c-4172-bcc6-ee0b48cb221a service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:52.172383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d591ff7-743c-4172-bcc6-ee0b48cb221a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:52.172383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-2d591ff7-743c-4172-bcc6-ee0b48cb221a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:52.232112 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d591ff7-743c-4172-bcc6-ee0b48cb221a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:52.233986 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d591ff7-743c-4172-bcc6-ee0b48cb221a service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:52.247432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1322/2636] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:31:52.391866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:52.577965 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eed6cd-1081-4ea8-8733-4cc38569b03a service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:52.579666 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-18eed6cd-1081-4ea8-8733-4cc38569b03a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:52.595997 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eed6cd-1081-4ea8-8733-4cc38569b03a service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:52.602318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05fbb7f-7f82-4f0c-bcd1-b8721551ac11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:52.603622 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05fbb7f-7f82-4f0c-bcd1-b8721551ac11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:52.604157 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05fbb7f-7f82-4f0c-bcd1-b8721551ac11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:52.605192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05fbb7f-7f82-4f0c-bcd1-b8721551ac11 admin admin] PATH_INFO: `/v3/projects/8f1a568e94c947c9bcd1f121bb11bc63` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:52.607783 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05fbb7f-7f82-4f0c-bcd1-b8721551ac11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f1a568e94c947c9bcd1f121bb11bc63)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:52.610133 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05fbb7f-7f82-4f0c-bcd1-b8721551ac11 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:52.850650 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e05fbb7f-7f82-4f0c-bcd1-b8721551ac11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f1a568e94c947c9bcd1f121bb11bc63', 'request_id': 'req-e05fbb7f-7f82-4f0c-bcd1-b8721551ac11'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:52.857617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e05fbb7f-7f82-4f0c-bcd1-b8721551ac11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f1a568e94c947c9bcd1f121bb11bc63', 'request_id': 'req-e05fbb7f-7f82-4f0c-bcd1-b8721551ac11'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:52.863302 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e05fbb7f-7f82-4f0c-bcd1-b8721551ac11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f1a568e94c947c9bcd1f121bb11bc63', 'request_id': 'req-e05fbb7f-7f82-4f0c-bcd1-b8721551ac11'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:52.990838 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e05fbb7f-7f82-4f0c-bcd1-b8721551ac11 admin admin] The token cache is being invalidate because project 8f1a568e94c947c9bcd1f121bb11bc63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:31:52.991161 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e05fbb7f-7f82-4f0c-bcd1-b8721551ac11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:52.991870 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-e05fbb7f-7f82-4f0c-bcd1-b8721551ac11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:52.993445 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1315/2637] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:31:52 2022] DELETE /identity/v3/projects/8f1a568e94c947c9bcd1f121bb11bc63 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:52.997380 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d591ff7-743c-4172-bcc6-ee0b48cb221a service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:53.008490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d591ff7-743c-4172-bcc6-ee0b48cb221a service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:53.066579 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e3de9-03aa-490a-a7c2-3a18d65f341d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:53.067433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e3de9-03aa-490a-a7c2-3a18d65f341d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:53.067716 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e3de9-03aa-490a-a7c2-3a18d65f341d admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:53.067992 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e3de9-03aa-490a-a7c2-3a18d65f341d admin admin] PATH_INFO: `/v3/users/002ddead00fd47ba93b2b111bb1f90d3` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:53.069190 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e3de9-03aa-490a-a7c2-3a18d65f341d admin admin] RBAC: Authorizing `identity:delete_user(user_id=002ddead00fd47ba93b2b111bb1f90d3)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:53.071768 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e3de9-03aa-490a-a7c2-3a18d65f341d admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:53.415107 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-361e3de9-03aa-490a-a7c2-3a18d65f341d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '002ddead00fd47ba93b2b111bb1f90d3'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:53.436030 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-361e3de9-03aa-490a-a7c2-3a18d65f341d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '002ddead00fd47ba93b2b111bb1f90d3'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:53.436815 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-361e3de9-03aa-490a-a7c2-3a18d65f341d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '002ddead00fd47ba93b2b111bb1f90d3'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:53.437442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-361e3de9-03aa-490a-a7c2-3a18d65f341d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '002ddead00fd47ba93b2b111bb1f90d3'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:53.468013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-361e3de9-03aa-490a-a7c2-3a18d65f341d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '002ddead00fd47ba93b2b111bb1f90d3'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:53.599464 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1323/2638] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:31:52 2022] DELETE /identity/v3/users/002ddead00fd47ba93b2b111bb1f90d3 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:53.820061 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:54.245273 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05fbb7f-7f82-4f0c-bcd1-b8721551ac11 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:54.247125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e05fbb7f-7f82-4f0c-bcd1-b8721551ac11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:54.280701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05fbb7f-7f82-4f0c-bcd1-b8721551ac11 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:54.341343 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636503e1-5c09-479c-b8d6-c551d521f2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:54.342230 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636503e1-5c09-479c-b8d6-c551d521f2c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:54.342522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636503e1-5c09-479c-b8d6-c551d521f2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:54.342810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636503e1-5c09-479c-b8d6-c551d521f2c3 admin admin] PATH_INFO: `/v3/projects/ebef946ffbac49cfa8c2172ccfd75795` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:54.344076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636503e1-5c09-479c-b8d6-c551d521f2c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebef946ffbac49cfa8c2172ccfd75795)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:54.346054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636503e1-5c09-479c-b8d6-c551d521f2c3 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:54.568128 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-636503e1-5c09-479c-b8d6-c551d521f2c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebef946ffbac49cfa8c2172ccfd75795', 'request_id': 'req-636503e1-5c09-479c-b8d6-c551d521f2c3'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:54.575425 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-636503e1-5c09-479c-b8d6-c551d521f2c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebef946ffbac49cfa8c2172ccfd75795', 'request_id': 'req-636503e1-5c09-479c-b8d6-c551d521f2c3'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:54.582479 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-636503e1-5c09-479c-b8d6-c551d521f2c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebef946ffbac49cfa8c2172ccfd75795', 'request_id': 'req-636503e1-5c09-479c-b8d6-c551d521f2c3'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:54.596486 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:54.755986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-636503e1-5c09-479c-b8d6-c551d521f2c3 admin admin] The token cache is being invalidate because project ebef946ffbac49cfa8c2172ccfd75795 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:31:54.756198 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-636503e1-5c09-479c-b8d6-c551d521f2c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:54.756827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-636503e1-5c09-479c-b8d6-c551d521f2c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:54.758963 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1316/2639] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:31:54 2022] DELETE /identity/v3/projects/ebef946ffbac49cfa8c2172ccfd75795 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:54.764984 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a5fa6-c5fc-4548-9a83-507d02a2529c None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:54.765276 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a5fa6-c5fc-4548-9a83-507d02a2529c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:54.765552 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a5fa6-c5fc-4548-9a83-507d02a2529c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:54.785530 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-4c9a5fa6-c5fc-4548-9a83-507d02a2529c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:54.786743 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-4c9a5fa6-c5fc-4548-9a83-507d02a2529c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:54.849382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1324/2640] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:54.857321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d533b5-c04b-4222-83c3-5d13d4a0451c None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:54.858120 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d533b5-c04b-4222-83c3-5d13d4a0451c None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:54.858444 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d533b5-c04b-4222-83c3-5d13d4a0451c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:54.878386 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-27d533b5-c04b-4222-83c3-5d13d4a0451c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:54.879630 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-27d533b5-c04b-4222-83c3-5d13d4a0451c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:54.888389 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1317/2641] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:31:54.893304 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a5fa6-c5fc-4548-9a83-507d02a2529c None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:54.902985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a5fa6-c5fc-4548-9a83-507d02a2529c None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:54.907528 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f034af-e2ab-47b3-9cc4-8ab0b32fd544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:54.908239 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f034af-e2ab-47b3-9cc4-8ab0b32fd544 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:54.908450 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f034af-e2ab-47b3-9cc4-8ab0b32fd544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:54.908723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f034af-e2ab-47b3-9cc4-8ab0b32fd544 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:54.909524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f034af-e2ab-47b3-9cc4-8ab0b32fd544 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:31:54.909826 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f034af-e2ab-47b3-9cc4-8ab0b32fd544 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:54.911534 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f034af-e2ab-47b3-9cc4-8ab0b32fd544 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:54.926445 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1325/2642] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:31:54.931669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d533b5-c04b-4222-83c3-5d13d4a0451c None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:54.931669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-27d533b5-c04b-4222-83c3-5d13d4a0451c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:54.934488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f034af-e2ab-47b3-9cc4-8ab0b32fd544 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:54.944147 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d533b5-c04b-4222-83c3-5d13d4a0451c None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:54.944613 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f034af-e2ab-47b3-9cc4-8ab0b32fd544 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:54.949617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83120ac4-6ea1-4336-99c4-63e8dde0811b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:54.949792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe508c-f7a5-4d1d-b385-31345a259007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:54.950390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe508c-f7a5-4d1d-b385-31345a259007 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:54.950703 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe508c-f7a5-4d1d-b385-31345a259007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:54.950948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83120ac4-6ea1-4336-99c4-63e8dde0811b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:54.951076 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe508c-f7a5-4d1d-b385-31345a259007 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:54.951242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83120ac4-6ea1-4336-99c4-63e8dde0811b admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:54.951522 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83120ac4-6ea1-4336-99c4-63e8dde0811b admin admin] PATH_INFO: `/v3/users/57ad18ab796f494e848529f114245ea6` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:54.951672 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe508c-f7a5-4d1d-b385-31345a259007 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:54.952730 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83120ac4-6ea1-4336-99c4-63e8dde0811b admin admin] RBAC: Authorizing `identity:delete_user(user_id=57ad18ab796f494e848529f114245ea6)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:54.953606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe508c-f7a5-4d1d-b385-31345a259007 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:54.954619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83120ac4-6ea1-4336-99c4-63e8dde0811b admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:55.084139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1326/2643] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:31:54 2022] POST /identity/v3/projects => generated 387 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:31:55.085331 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe508c-f7a5-4d1d-b385-31345a259007 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:55.168851 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe508c-f7a5-4d1d-b385-31345a259007 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:55.173850 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-83120ac4-6ea1-4336-99c4-63e8dde0811b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57ad18ab796f494e848529f114245ea6'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:55.178210 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017efa2f-317c-4053-8d29-4892837be047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:55.178926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017efa2f-317c-4053-8d29-4892837be047 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:55.179245 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017efa2f-317c-4053-8d29-4892837be047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:55.180105 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017efa2f-317c-4053-8d29-4892837be047 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:55.180820 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017efa2f-317c-4053-8d29-4892837be047 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:55.182779 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017efa2f-317c-4053-8d29-4892837be047 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:55.183812 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-83120ac4-6ea1-4336-99c4-63e8dde0811b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57ad18ab796f494e848529f114245ea6'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:55.184502 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-83120ac4-6ea1-4336-99c4-63e8dde0811b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57ad18ab796f494e848529f114245ea6'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:55.185577 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-83120ac4-6ea1-4336-99c4-63e8dde0811b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57ad18ab796f494e848529f114245ea6'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:55.198815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-83120ac4-6ea1-4336-99c4-63e8dde0811b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57ad18ab796f494e848529f114245ea6'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:55.305902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1327/2644] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:31:55 2022] POST /identity/v3/users => generated 294 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:55.307112 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1318/2645] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:31:54 2022] DELETE /identity/v3/users/57ad18ab796f494e848529f114245ea6 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:31:55.309983 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017efa2f-317c-4053-8d29-4892837be047 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:55.311468 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0552b-1f71-4beb-ac89-1df130c75547 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:55.315471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-017efa2f-317c-4053-8d29-4892837be047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:55.315615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0552b-1f71-4beb-ac89-1df130c75547 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:55.315615 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0552b-1f71-4beb-ac89-1df130c75547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:55.325706 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017efa2f-317c-4053-8d29-4892837be047 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:55.329113 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-a6e0552b-1f71-4beb-ac89-1df130c75547 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:55.330305 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-a6e0552b-1f71-4beb-ac89-1df130c75547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:55.388028 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3804d8-f9e4-4898-ae5a-5a947c6bbe96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:55.389306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3804d8-f9e4-4898-ae5a-5a947c6bbe96 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:55.389661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3804d8-f9e4-4898-ae5a-5a947c6bbe96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:55.390194 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3804d8-f9e4-4898-ae5a-5a947c6bbe96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:55.391489 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3804d8-f9e4-4898-ae5a-5a947c6bbe96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:55.395447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3804d8-f9e4-4898-ae5a-5a947c6bbe96 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:55.395584 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1319/2646] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:31:55.403933 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0552b-1f71-4beb-ac89-1df130c75547 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:55.404200 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1328/2647] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:31:55.405553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-a6e0552b-1f71-4beb-ac89-1df130c75547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:55.408088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3804d8-f9e4-4898-ae5a-5a947c6bbe96 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:55.418093 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3804d8-f9e4-4898-ae5a-5a947c6bbe96 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:55.422949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa199d4c-9246-4d41-9ccd-ee726df22bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:55.423604 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa199d4c-9246-4d41-9ccd-ee726df22bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:55.423917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa199d4c-9246-4d41-9ccd-ee726df22bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:55.424176 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa199d4c-9246-4d41-9ccd-ee726df22bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:55.424901 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa199d4c-9246-4d41-9ccd-ee726df22bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:55.426510 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa199d4c-9246-4d41-9ccd-ee726df22bd0 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:55.436524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1329/2648] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:55.440720 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa199d4c-9246-4d41-9ccd-ee726df22bd0 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:55.451672 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa199d4c-9246-4d41-9ccd-ee726df22bd0 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:55.457576 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e9921-88d8-476b-87c7-4267659ece7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:55.458510 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e9921-88d8-476b-87c7-4267659ece7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:55.462233 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e9921-88d8-476b-87c7-4267659ece7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:55.462233 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e9921-88d8-476b-87c7-4267659ece7a admin admin] PATH_INFO: `/v3/projects/9541069573d04541b3d567f3c06674df/users/ccb1c86f20ff4e83976cae0a8bc58936/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:55.464506 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0552b-1f71-4beb-ac89-1df130c75547 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:55.468193 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3aee8-6bb2-409a-ae6f-a85bdae0fa8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:55.468846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3aee8-6bb2-409a-ae6f-a85bdae0fa8b service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:55.469134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3aee8-6bb2-409a-ae6f-a85bdae0fa8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:55.469414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3aee8-6bb2-409a-ae6f-a85bdae0fa8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:55.474502 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e9921-88d8-476b-87c7-4267659ece7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9541069573d04541b3d567f3c06674df, user_id=ccb1c86f20ff4e83976cae0a8bc58936, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:55.477273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e9921-88d8-476b-87c7-4267659ece7a admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:55.478693 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3aee8-6bb2-409a-ae6f-a85bdae0fa8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:55.480161 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3aee8-6bb2-409a-ae6f-a85bdae0fa8b service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:55.489873 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1320/2649] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:31:55.563591 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1330/2650] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:55 2022] PUT /identity/v3/projects/9541069573d04541b3d567f3c06674df/users/ccb1c86f20ff4e83976cae0a8bc58936/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:31:55.575418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c0124-c501-4bde-8a05-20606e551a67 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:55.575857 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c0124-c501-4bde-8a05-20606e551a67 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:55.576106 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c0124-c501-4bde-8a05-20606e551a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:55.601803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-d88c0124-c501-4bde-8a05-20606e551a67 None None] MFA Rules not processed for user `ccb1c86f20ff4e83976cae0a8bc58936`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:55.603090 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-d88c0124-c501-4bde-8a05-20606e551a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:55.669287 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1321/2651] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:55.673710 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e9921-88d8-476b-87c7-4267659ece7a admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:55.734311 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e9921-88d8-476b-87c7-4267659ece7a admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:55.742789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1f7aa-2037-441b-822f-c84144fbf794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:55.742789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1f7aa-2037-441b-822f-c84144fbf794 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:55.742789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1f7aa-2037-441b-822f-c84144fbf794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:55.742789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1f7aa-2037-441b-822f-c84144fbf794 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:55.743256 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1f7aa-2037-441b-822f-c84144fbf794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:55.744308 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1f7aa-2037-441b-822f-c84144fbf794 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:55.892024 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1331/2652] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:31:55 2022] POST /identity/v3/projects => generated 387 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:55.893256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c0124-c501-4bde-8a05-20606e551a67 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:55.904090 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c0124-c501-4bde-8a05-20606e551a67 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:55.922028 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:55.959181 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e1dbe-44d9-42eb-8225-82e9f7615fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:55.959857 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e1dbe-44d9-42eb-8225-82e9f7615fde admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:55.960152 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e1dbe-44d9-42eb-8225-82e9f7615fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:55.960429 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e1dbe-44d9-42eb-8225-82e9f7615fde admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:55.961143 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1e1dbe-44d9-42eb-8225-82e9f7615fde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:55.963084 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1e1dbe-44d9-42eb-8225-82e9f7615fde admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:56.018276 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1f7aa-2037-441b-822f-c84144fbf794 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:56.018276 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-8bf1f7aa-2037-441b-822f-c84144fbf794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:56.029622 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1f7aa-2037-441b-822f-c84144fbf794 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:56.034580 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afbb0e-f5b6-45e1-90c9-450dd83c5336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:56.035548 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afbb0e-f5b6-45e1-90c9-450dd83c5336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:56.035858 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afbb0e-f5b6-45e1-90c9-450dd83c5336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:56.036193 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afbb0e-f5b6-45e1-90c9-450dd83c5336 admin admin] PATH_INFO: `/v3/projects/96ddf3fb81d8436d81fa2d7026749369` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:56.037564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26afbb0e-f5b6-45e1-90c9-450dd83c5336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96ddf3fb81d8436d81fa2d7026749369)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:56.040420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26afbb0e-f5b6-45e1-90c9-450dd83c5336 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:56.111106 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1322/2653] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:31:55 2022] POST /identity/v3/users => generated 294 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:56.115438 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e1dbe-44d9-42eb-8225-82e9f7615fde admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:56.125562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e1dbe-44d9-42eb-8225-82e9f7615fde admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:56.130231 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb9c7e-fefa-4ac7-8b60-d7dfa425b5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:56.130871 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb9c7e-fefa-4ac7-8b60-d7dfa425b5cb admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:56.131191 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb9c7e-fefa-4ac7-8b60-d7dfa425b5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:56.131469 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb9c7e-fefa-4ac7-8b60-d7dfa425b5cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:56.132184 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb9c7e-fefa-4ac7-8b60-d7dfa425b5cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:56.133801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb9c7e-fefa-4ac7-8b60-d7dfa425b5cb admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:56.143690 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1323/2654] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:31:56.149654 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb9c7e-fefa-4ac7-8b60-d7dfa425b5cb admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:56.159863 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb9c7e-fefa-4ac7-8b60-d7dfa425b5cb admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:56.165290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d59ca-76df-444f-badf-850147e6cf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:56.165967 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d59ca-76df-444f-badf-850147e6cf80 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:56.166257 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d59ca-76df-444f-badf-850147e6cf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:56.166537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d59ca-76df-444f-badf-850147e6cf80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:56.167514 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d59ca-76df-444f-badf-850147e6cf80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:56.169008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d59ca-76df-444f-badf-850147e6cf80 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:56.182902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1324/2655] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:56.191271 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d59ca-76df-444f-badf-850147e6cf80 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:56.202155 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d59ca-76df-444f-badf-850147e6cf80 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:56.203311 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-26afbb0e-f5b6-45e1-90c9-450dd83c5336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96ddf3fb81d8436d81fa2d7026749369', 'request_id': 'req-26afbb0e-f5b6-45e1-90c9-450dd83c5336'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:56.214495 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-26afbb0e-f5b6-45e1-90c9-450dd83c5336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96ddf3fb81d8436d81fa2d7026749369', 'request_id': 'req-26afbb0e-f5b6-45e1-90c9-450dd83c5336'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:56.264621 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff5a570-1f07-45f9-8a6f-ccad8c4df836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:56.266875 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5a570-1f07-45f9-8a6f-ccad8c4df836 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:56.266875 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5a570-1f07-45f9-8a6f-ccad8c4df836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:56.266875 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5a570-1f07-45f9-8a6f-ccad8c4df836 admin admin] PATH_INFO: `/v3/projects/db063cba941b47faace2dfc86e9885b5/users/255c8ecc45724283a4977ffcceca70f7/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:56.283292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff5a570-1f07-45f9-8a6f-ccad8c4df836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db063cba941b47faace2dfc86e9885b5, user_id=255c8ecc45724283a4977ffcceca70f7, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:56.286860 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff5a570-1f07-45f9-8a6f-ccad8c4df836 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:56.322492 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-26afbb0e-f5b6-45e1-90c9-450dd83c5336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96ddf3fb81d8436d81fa2d7026749369', 'request_id': 'req-26afbb0e-f5b6-45e1-90c9-450dd83c5336'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:56.344741 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-26afbb0e-f5b6-45e1-90c9-450dd83c5336 admin admin] The token cache is being invalidate because project 96ddf3fb81d8436d81fa2d7026749369 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:31:56.345029 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-26afbb0e-f5b6-45e1-90c9-450dd83c5336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:56.345751 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-26afbb0e-f5b6-45e1-90c9-450dd83c5336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:56.347721 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1332/2656] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:31:56 2022] DELETE /identity/v3/projects/96ddf3fb81d8436d81fa2d7026749369 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:56.351004 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:56.355327 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afbb0e-f5b6-45e1-90c9-450dd83c5336 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:56.358986 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-26afbb0e-f5b6-45e1-90c9-450dd83c5336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:56.369949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afbb0e-f5b6-45e1-90c9-450dd83c5336 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:56.374541 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7973a-0315-41fe-954b-0b4e895a45b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:56.375443 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7973a-0315-41fe-954b-0b4e895a45b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:56.375727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7973a-0315-41fe-954b-0b4e895a45b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:56.376002 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7973a-0315-41fe-954b-0b4e895a45b0 admin admin] PATH_INFO: `/v3/users/50f248be1d31469882376ed6f3b29555` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:56.377202 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7973a-0315-41fe-954b-0b4e895a45b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50f248be1d31469882376ed6f3b29555)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:56.379197 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7973a-0315-41fe-954b-0b4e895a45b0 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:56.442599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1325/2657] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:56 2022] PUT /identity/v3/projects/db063cba941b47faace2dfc86e9885b5/users/255c8ecc45724283a4977ffcceca70f7/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:31:56.452928 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b59de-7ec7-4152-9b02-6d2870fa71ca None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:56.453381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b59de-7ec7-4152-9b02-6d2870fa71ca None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:56.453841 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b59de-7ec7-4152-9b02-6d2870fa71ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:56.481202 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-fe1b59de-7ec7-4152-9b02-6d2870fa71ca None None] MFA Rules not processed for user `255c8ecc45724283a4977ffcceca70f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:56.482432 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-fe1b59de-7ec7-4152-9b02-6d2870fa71ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:56.547877 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1326/2658] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:56.554774 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b59de-7ec7-4152-9b02-6d2870fa71ca None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:56.556409 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-fe1b59de-7ec7-4152-9b02-6d2870fa71ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:56.625329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b59de-7ec7-4152-9b02-6d2870fa71ca None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:56.629854 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e3c59-48b0-4b23-9b58-dbf450fb2352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:56.630525 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e3c59-48b0-4b23-9b58-dbf450fb2352 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:56.630815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e3c59-48b0-4b23-9b58-dbf450fb2352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:56.631128 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e3c59-48b0-4b23-9b58-dbf450fb2352 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:56.631835 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9e3c59-48b0-4b23-9b58-dbf450fb2352 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:56.634254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9e3c59-48b0-4b23-9b58-dbf450fb2352 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:56.667077 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-aca7973a-0315-41fe-954b-0b4e895a45b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50f248be1d31469882376ed6f3b29555'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:56.674151 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-aca7973a-0315-41fe-954b-0b4e895a45b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50f248be1d31469882376ed6f3b29555'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:56.678246 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-aca7973a-0315-41fe-954b-0b4e895a45b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50f248be1d31469882376ed6f3b29555'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:56.678865 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-aca7973a-0315-41fe-954b-0b4e895a45b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50f248be1d31469882376ed6f3b29555'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:56.691297 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-aca7973a-0315-41fe-954b-0b4e895a45b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50f248be1d31469882376ed6f3b29555'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:56.749699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1327/2659] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:31:56 2022] POST /identity/v3/projects => generated 385 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:31:56.753197 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e3c59-48b0-4b23-9b58-dbf450fb2352 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:56.755176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-aa9e3c59-48b0-4b23-9b58-dbf450fb2352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:56.832669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1333/2660] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:31:56 2022] DELETE /identity/v3/users/50f248be1d31469882376ed6f3b29555 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:56.832897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e3c59-48b0-4b23-9b58-dbf450fb2352 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:56.911207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0deb9-4ff8-41b0-a631-d612743ca89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:56.911892 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0deb9-4ff8-41b0-a631-d612743ca89a admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:56.912147 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0deb9-4ff8-41b0-a631-d612743ca89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:56.912584 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0deb9-4ff8-41b0-a631-d612743ca89a admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:56.913316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0deb9-4ff8-41b0-a631-d612743ca89a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:56.915433 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0deb9-4ff8-41b0-a631-d612743ca89a admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:57.051218 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1328/2661] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:31:56 2022] POST /identity/v3/users => generated 299 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:57.052464 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7973a-0315-41fe-954b-0b4e895a45b0 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:57.063123 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7973a-0315-41fe-954b-0b4e895a45b0 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:57.082836 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a628fbd-7c49-4fba-b114-9dc53cec802b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:57.083568 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a628fbd-7c49-4fba-b114-9dc53cec802b admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:57.083865 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a628fbd-7c49-4fba-b114-9dc53cec802b admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:57.084149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a628fbd-7c49-4fba-b114-9dc53cec802b admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:57.084902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a628fbd-7c49-4fba-b114-9dc53cec802b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:57.086618 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a628fbd-7c49-4fba-b114-9dc53cec802b admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:57.099505 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1334/2662] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:57.116398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0deb9-4ff8-41b0-a631-d612743ca89a admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:57.127882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0deb9-4ff8-41b0-a631-d612743ca89a admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:57.133914 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc06bab-5320-4fa4-b9ca-c46d9997b563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:57.134828 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc06bab-5320-4fa4-b9ca-c46d9997b563 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:57.135172 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc06bab-5320-4fa4-b9ca-c46d9997b563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:57.135596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc06bab-5320-4fa4-b9ca-c46d9997b563 admin admin] PATH_INFO: `/v3/projects/2938e65d9cb34242a8bbac7c16ba6d6b/users/403f03b9ec044e8781881fe24b2c84fe/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:57.159379 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc06bab-5320-4fa4-b9ca-c46d9997b563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2938e65d9cb34242a8bbac7c16ba6d6b, user_id=403f03b9ec044e8781881fe24b2c84fe, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:57.159379 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc06bab-5320-4fa4-b9ca-c46d9997b563 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:57.202588 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:57.254300 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1329/2663] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:57 2022] PUT /identity/v3/projects/2938e65d9cb34242a8bbac7c16ba6d6b/users/403f03b9ec044e8781881fe24b2c84fe/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:57.258828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a628fbd-7c49-4fba-b114-9dc53cec802b admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:57.270753 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a628fbd-7c49-4fba-b114-9dc53cec802b admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:57.324326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc06bab-5320-4fa4-b9ca-c46d9997b563 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:57.326264 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-4bc06bab-5320-4fa4-b9ca-c46d9997b563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:57.329007 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad221445-d5de-4325-a54d-cad04a689f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:57.329687 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad221445-d5de-4325-a54d-cad04a689f45 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:57.329976 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad221445-d5de-4325-a54d-cad04a689f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:57.330261 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad221445-d5de-4325-a54d-cad04a689f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:57.331022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad221445-d5de-4325-a54d-cad04a689f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:57.332704 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad221445-d5de-4325-a54d-cad04a689f45 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:57.339850 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc06bab-5320-4fa4-b9ca-c46d9997b563 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:57.343935 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1335/2664] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:57.348058 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad221445-d5de-4325-a54d-cad04a689f45 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:57.348541 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fa834-a8b1-4e4a-9016-ebdbca8b036d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:57.349587 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fa834-a8b1-4e4a-9016-ebdbca8b036d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:57.349986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fa834-a8b1-4e4a-9016-ebdbca8b036d admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:57.350273 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fa834-a8b1-4e4a-9016-ebdbca8b036d admin admin] PATH_INFO: `/v3/projects/37249065f80a4097bbdad538cf58d59d` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:57.352048 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fa834-a8b1-4e4a-9016-ebdbca8b036d admin admin] RBAC: Authorizing `identity:delete_project(project_id=37249065f80a4097bbdad538cf58d59d)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:57.353982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fa834-a8b1-4e4a-9016-ebdbca8b036d admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:57.364206 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad221445-d5de-4325-a54d-cad04a689f45 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:57.371185 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8543f81-36f9-4d86-a485-00c93017d212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:57.372191 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8543f81-36f9-4d86-a485-00c93017d212 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:57.372877 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8543f81-36f9-4d86-a485-00c93017d212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:57.373316 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8543f81-36f9-4d86-a485-00c93017d212 admin admin] PATH_INFO: `/v3/projects/2938e65d9cb34242a8bbac7c16ba6d6b/users/403f03b9ec044e8781881fe24b2c84fe/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:57.375592 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8543f81-36f9-4d86-a485-00c93017d212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2938e65d9cb34242a8bbac7c16ba6d6b, user_id=403f03b9ec044e8781881fe24b2c84fe, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:57.379340 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8543f81-36f9-4d86-a485-00c93017d212 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:57.392854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1336/2665] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:57 2022] PUT /identity/v3/projects/2938e65d9cb34242a8bbac7c16ba6d6b/users/403f03b9ec044e8781881fe24b2c84fe/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:31:57.402493 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ecf3c-d727-43a3-ade9-e8d0e0fd3ecf None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:57.402786 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ecf3c-d727-43a3-ade9-e8d0e0fd3ecf None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:57.403177 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ecf3c-d727-43a3-ade9-e8d0e0fd3ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:57.432192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:57.432744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-448ecf3c-d727-43a3-ade9-e8d0e0fd3ecf None None] MFA Rules not processed for user `403f03b9ec044e8781881fe24b2c84fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:57.434126 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-448ecf3c-d727-43a3-ade9-e8d0e0fd3ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:57.473003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ab3fa834-a8b1-4e4a-9016-ebdbca8b036d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37249065f80a4097bbdad538cf58d59d', 'request_id': 'req-ab3fa834-a8b1-4e4a-9016-ebdbca8b036d'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:57.480780 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ab3fa834-a8b1-4e4a-9016-ebdbca8b036d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37249065f80a4097bbdad538cf58d59d', 'request_id': 'req-ab3fa834-a8b1-4e4a-9016-ebdbca8b036d'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:57.486858 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ab3fa834-a8b1-4e4a-9016-ebdbca8b036d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37249065f80a4097bbdad538cf58d59d', 'request_id': 'req-ab3fa834-a8b1-4e4a-9016-ebdbca8b036d'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:57.543124 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1337/2666] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:31:57.552497 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ecf3c-d727-43a3-ade9-e8d0e0fd3ecf None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:57.553924 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-448ecf3c-d727-43a3-ade9-e8d0e0fd3ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:57.590748 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ab3fa834-a8b1-4e4a-9016-ebdbca8b036d admin admin] The token cache is being invalidate because project 37249065f80a4097bbdad538cf58d59d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:31:57.591089 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ab3fa834-a8b1-4e4a-9016-ebdbca8b036d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:57.591926 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ab3fa834-a8b1-4e4a-9016-ebdbca8b036d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:31:57.593909 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1330/2667] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:31:57 2022] DELETE /identity/v3/projects/37249065f80a4097bbdad538cf58d59d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:31:57.600548 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd8e5f-f360-444b-8651-ac89d63c76e0 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:57.600842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd8e5f-f360-444b-8651-ac89d63c76e0 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:57.601127 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd8e5f-f360-444b-8651-ac89d63c76e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:57.620627 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ecf3c-d727-43a3-ade9-e8d0e0fd3ecf None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:57.621040 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-a4dd8e5f-f360-444b-8651-ac89d63c76e0 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:57.622256 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-a4dd8e5f-f360-444b-8651-ac89d63c76e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:57.670399 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61ce0c-dcda-4d0c-85b5-449702eaa8a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:57.671100 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61ce0c-dcda-4d0c-85b5-449702eaa8a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:57.671392 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61ce0c-dcda-4d0c-85b5-449702eaa8a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:57.671675 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61ce0c-dcda-4d0c-85b5-449702eaa8a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:57.673475 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-fd61ce0c-dcda-4d0c-85b5-449702eaa8a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:57.681209 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1331/2668] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:57.688882 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e8ac6-cf7a-4e8c-afbc-b6d9123052a4 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:57.696064 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e8ac6-cf7a-4e8c-afbc-b6d9123052a4 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:57.696064 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e8ac6-cf7a-4e8c-afbc-b6d9123052a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:57.707991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-239e8ac6-cf7a-4e8c-afbc-b6d9123052a4 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:57.709208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-239e8ac6-cf7a-4e8c-afbc-b6d9123052a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:57.722973 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1332/2669] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:31:57.730258 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e8ac6-cf7a-4e8c-afbc-b6d9123052a4 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:57.737761 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e8ac6-cf7a-4e8c-afbc-b6d9123052a4 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:57.744428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ea0f6-ac37-4298-a27f-ccfa56f238b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:57.745655 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ea0f6-ac37-4298-a27f-ccfa56f238b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:57.746055 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ea0f6-ac37-4298-a27f-ccfa56f238b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:57.746309 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ea0f6-ac37-4298-a27f-ccfa56f238b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:57.747088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ea0f6-ac37-4298-a27f-ccfa56f238b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:31:57.747449 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ea0f6-ac37-4298-a27f-ccfa56f238b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:57.747936 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd61ce0c-dcda-4d0c-85b5-449702eaa8a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:57.753223 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ea0f6-ac37-4298-a27f-ccfa56f238b8 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:57.753360 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd61ce0c-dcda-4d0c-85b5-449702eaa8a5 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:57.760354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1338/2670] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:31:57.765346 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1333/2671] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:31:57.770086 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61ce0c-dcda-4d0c-85b5-449702eaa8a5 service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:57.788287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61ce0c-dcda-4d0c-85b5-449702eaa8a5 service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:57.793270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c2551-73cc-44fd-bbd4-04a61916194d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:57.793937 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c2551-73cc-44fd-bbd4-04a61916194d admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:57.794229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c2551-73cc-44fd-bbd4-04a61916194d admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:57.794509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c2551-73cc-44fd-bbd4-04a61916194d admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:57.795466 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c2551-73cc-44fd-bbd4-04a61916194d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:57.797256 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c2551-73cc-44fd-bbd4-04a61916194d admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:57.952550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1339/2672] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:31:57 2022] POST /identity/v3/projects => generated 365 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:57.957113 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ea0f6-ac37-4298-a27f-ccfa56f238b8 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:57.967207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ea0f6-ac37-4298-a27f-ccfa56f238b8 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:58.025670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c5201-c2eb-4031-910e-d34042f6201b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:58.030039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c5201-c2eb-4031-910e-d34042f6201b admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:58.030039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c5201-c2eb-4031-910e-d34042f6201b admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:58.030039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c5201-c2eb-4031-910e-d34042f6201b admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:58.030039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c5201-c2eb-4031-910e-d34042f6201b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:58.030039 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c5201-c2eb-4031-910e-d34042f6201b admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:58.197552 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1334/2673] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:31:57 2022] POST /identity/v3/users => generated 283 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:58.201587 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c2551-73cc-44fd-bbd4-04a61916194d admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:58.213691 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c2551-73cc-44fd-bbd4-04a61916194d admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:58.218565 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677ec5c-b738-4bfe-9528-43ad67796f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:58.219269 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677ec5c-b738-4bfe-9528-43ad67796f08 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:58.219589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677ec5c-b738-4bfe-9528-43ad67796f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:58.219835 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677ec5c-b738-4bfe-9528-43ad67796f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:58.220580 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677ec5c-b738-4bfe-9528-43ad67796f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:58.222214 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677ec5c-b738-4bfe-9528-43ad67796f08 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:58.242644 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1340/2674] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:58.243845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c5201-c2eb-4031-910e-d34042f6201b admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:58.254408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c5201-c2eb-4031-910e-d34042f6201b admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:58.259533 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b7e77-2468-4799-97c0-c1d01b6017df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:58.260355 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b7e77-2468-4799-97c0-c1d01b6017df admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:58.260657 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b7e77-2468-4799-97c0-c1d01b6017df admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:58.260939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b7e77-2468-4799-97c0-c1d01b6017df admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:58.261710 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b7e77-2468-4799-97c0-c1d01b6017df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:58.263376 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b7e77-2468-4799-97c0-c1d01b6017df admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:58.276812 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1335/2675] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:58.277907 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677ec5c-b738-4bfe-9528-43ad67796f08 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:58.288096 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677ec5c-b738-4bfe-9528-43ad67796f08 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:58.294018 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247e23b-4af5-48a5-b02d-3d0f64e6128b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:58.294946 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247e23b-4af5-48a5-b02d-3d0f64e6128b admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:58.295565 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247e23b-4af5-48a5-b02d-3d0f64e6128b admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:58.295857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247e23b-4af5-48a5-b02d-3d0f64e6128b admin admin] PATH_INFO: `/v3/projects/3000869310b14b71ae077d3ecb73376c/users/5c2ebc8ef0884ab1940ccaa4a33af156/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:58.312758 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247e23b-4af5-48a5-b02d-3d0f64e6128b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3000869310b14b71ae077d3ecb73376c, user_id=5c2ebc8ef0884ab1940ccaa4a33af156, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:58.315470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247e23b-4af5-48a5-b02d-3d0f64e6128b admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:58.478330 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1341/2676] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:58 2022] PUT /identity/v3/projects/3000869310b14b71ae077d3ecb73376c/users/5c2ebc8ef0884ab1940ccaa4a33af156/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:31:58.488911 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af54780-a54c-44dd-a06f-3bf5c325d9dd None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:58.489204 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af54780-a54c-44dd-a06f-3bf5c325d9dd None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:58.489482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af54780-a54c-44dd-a06f-3bf5c325d9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:58.516177 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-7af54780-a54c-44dd-a06f-3bf5c325d9dd None None] MFA Rules not processed for user `5c2ebc8ef0884ab1940ccaa4a33af156`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:58.517372 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7af54780-a54c-44dd-a06f-3bf5c325d9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:58.587597 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1336/2677] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:58 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:31:58.590211 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247e23b-4af5-48a5-b02d-3d0f64e6128b admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:58.672520 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247e23b-4af5-48a5-b02d-3d0f64e6128b admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:58.677422 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2ba41-53a5-4877-9199-cf4da615afee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:58.678072 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2ba41-53a5-4877-9199-cf4da615afee admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:58.678366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2ba41-53a5-4877-9199-cf4da615afee admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:58.678646 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2ba41-53a5-4877-9199-cf4da615afee admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:58.680069 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2ba41-53a5-4877-9199-cf4da615afee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:58.682075 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2ba41-53a5-4877-9199-cf4da615afee admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:58.798806 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1342/2678] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:31:58 2022] POST /identity/v3/projects => generated 367 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:58.803004 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af54780-a54c-44dd-a06f-3bf5c325d9dd None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:58.814189 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af54780-a54c-44dd-a06f-3bf5c325d9dd None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:58.878845 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f83d8-9f47-4ba9-add7-227e2cb86e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:58.879637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f83d8-9f47-4ba9-add7-227e2cb86e8b admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:58.891170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f83d8-9f47-4ba9-add7-227e2cb86e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:58.891170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f83d8-9f47-4ba9-add7-227e2cb86e8b admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:58.891170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f83d8-9f47-4ba9-add7-227e2cb86e8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:58.891170 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f83d8-9f47-4ba9-add7-227e2cb86e8b admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:58.896986 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca243fe-3457-40b0-aabb-953150f684fb None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:58.897286 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca243fe-3457-40b0-aabb-953150f684fb None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:58.897568 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca243fe-3457-40b0-aabb-953150f684fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:58.916280 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-1ca243fe-3457-40b0-aabb-953150f684fb None None] MFA Rules not processed for user `86a640cdceb34bb9bfbb7b769e797d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:58.917479 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-1ca243fe-3457-40b0-aabb-953150f684fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:58.970500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1343/2679] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:31:59.054092 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1337/2680] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:31:58 2022] POST /identity/v3/users => generated 290 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:31:59.059005 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca243fe-3457-40b0-aabb-953150f684fb None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:59.067172 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:INSERT=5,keystone:SELECT=93,keystone:DELETE=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:59.071807 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca243fe-3457-40b0-aabb-953150f684fb None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:59.078213 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00567070-a51f-4f0d-8292-e22318cf0389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:59.078727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00567070-a51f-4f0d-8292-e22318cf0389 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:59.079047 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00567070-a51f-4f0d-8292-e22318cf0389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:59.079323 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00567070-a51f-4f0d-8292-e22318cf0389 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:59.080113 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00567070-a51f-4f0d-8292-e22318cf0389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:59.081961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00567070-a51f-4f0d-8292-e22318cf0389 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:59.092899 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1344/2681] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:31:59.096889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f83d8-9f47-4ba9-add7-227e2cb86e8b admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:59.107773 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f83d8-9f47-4ba9-add7-227e2cb86e8b admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:59.112472 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cf448-fa49-432a-8857-0f26ad70511b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:59.113631 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cf448-fa49-432a-8857-0f26ad70511b admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:59.113920 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cf448-fa49-432a-8857-0f26ad70511b admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:59.114203 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cf448-fa49-432a-8857-0f26ad70511b admin admin] PATH_INFO: `/v3/projects/187845dc71b6464c88a7f95902718ba1/users/7f679565035f4f50931699ece9cb4045/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:59.132950 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5cf448-fa49-432a-8857-0f26ad70511b admin admin] RBAC: Authorizing `identity:create_grant(project_id=187845dc71b6464c88a7f95902718ba1, user_id=7f679565035f4f50931699ece9cb4045, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:59.135678 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5cf448-fa49-432a-8857-0f26ad70511b admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:59.273232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1338/2682] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:59 2022] PUT /identity/v3/projects/187845dc71b6464c88a7f95902718ba1/users/7f679565035f4f50931699ece9cb4045/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:31:59.287160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00567070-a51f-4f0d-8292-e22318cf0389 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:59.298001 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00567070-a51f-4f0d-8292-e22318cf0389 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:59.344242 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cf448-fa49-432a-8857-0f26ad70511b admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:59.346028 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-4f5cf448-fa49-432a-8857-0f26ad70511b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:59.361688 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ebb68-9756-44bc-bf1a-dd918f91de24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:59.362511 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ebb68-9756-44bc-bf1a-dd918f91de24 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:59.362902 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ebb68-9756-44bc-bf1a-dd918f91de24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:59.363594 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ebb68-9756-44bc-bf1a-dd918f91de24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:59.364441 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ebb68-9756-44bc-bf1a-dd918f91de24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:59.366172 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ebb68-9756-44bc-bf1a-dd918f91de24 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:59.377409 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1345/2683] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:31:59.381134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ebb68-9756-44bc-bf1a-dd918f91de24 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:59.394629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ebb68-9756-44bc-bf1a-dd918f91de24 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:59.399656 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb06f0-fdcc-4e0c-a4d7-594283cc5024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:59.400589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb06f0-fdcc-4e0c-a4d7-594283cc5024 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:59.400875 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb06f0-fdcc-4e0c-a4d7-594283cc5024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:59.401162 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb06f0-fdcc-4e0c-a4d7-594283cc5024 admin admin] PATH_INFO: `/v3/projects/187845dc71b6464c88a7f95902718ba1/users/7f679565035f4f50931699ece9cb4045/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:59.403259 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb06f0-fdcc-4e0c-a4d7-594283cc5024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=187845dc71b6464c88a7f95902718ba1, user_id=7f679565035f4f50931699ece9cb4045, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:59.405906 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb06f0-fdcc-4e0c-a4d7-594283cc5024 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:59.409909 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:31:59.414243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cf448-fa49-432a-8857-0f26ad70511b admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:59.418231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1346/2684] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:31:59 2022] PUT /identity/v3/projects/187845dc71b6464c88a7f95902718ba1/users/7f679565035f4f50931699ece9cb4045/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:31:59.429100 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5d13e-c33f-47d4-a02e-e942cd347bc0 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:59.429965 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5d13e-c33f-47d4-a02e-e942cd347bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:59.430250 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5d13e-c33f-47d4-a02e-e942cd347bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:59.457968 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-43a5d13e-c33f-47d4-a02e-e942cd347bc0 None None] MFA Rules not processed for user `7f679565035f4f50931699ece9cb4045`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:31:59.459350 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-43a5d13e-c33f-47d4-a02e-e942cd347bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:59.466228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050d32b-5043-4d8d-a7f9-13ceca862559 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:59.466895 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050d32b-5043-4d8d-a7f9-13ceca862559 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:59.467211 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050d32b-5043-4d8d-a7f9-13ceca862559 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:59.467750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050d32b-5043-4d8d-a7f9-13ceca862559 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:59.469660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-f050d32b-5043-4d8d-a7f9-13ceca862559 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:59.534644 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1347/2685] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:31:59 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:31:59.535076 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f050d32b-5043-4d8d-a7f9-13ceca862559 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:59.536544 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f050d32b-5043-4d8d-a7f9-13ceca862559 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:59.546131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1339/2686] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:31:59.546643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5d13e-c33f-47d4-a02e-e942cd347bc0 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:59.548197 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-43a5d13e-c33f-47d4-a02e-e942cd347bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:31:59.560983 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5d13e-c33f-47d4-a02e-e942cd347bc0 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:59.564713 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf372c0-4d53-4b73-966b-3d96c4fc33ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:59.565368 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf372c0-4d53-4b73-966b-3d96c4fc33ff service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:59.565680 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf372c0-4d53-4b73-966b-3d96c4fc33ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:59.565985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf372c0-4d53-4b73-966b-3d96c4fc33ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:31:59.576134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf372c0-4d53-4b73-966b-3d96c4fc33ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:31:59.577702 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf372c0-4d53-4b73-966b-3d96c4fc33ff service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:31:59.587443 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1348/2687] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:59 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:31:59.979601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050d32b-5043-4d8d-a7f9-13ceca862559 service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:31:59.988787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050d32b-5043-4d8d-a7f9-13ceca862559 service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:31:59.993808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2da980-9091-44b4-b75b-04c6f94163e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:31:59.993808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2da980-9091-44b4-b75b-04c6f94163e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:31:59.993808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2da980-9091-44b4-b75b-04c6f94163e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:31:59.993808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2da980-9091-44b4-b75b-04c6f94163e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:00.063335 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2da980-9091-44b4-b75b-04c6f94163e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:00.064781 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2da980-9091-44b4-b75b-04c6f94163e5 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:00.074356 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1340/2688] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:31:59 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:32:00.243268 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:00.396534 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf372c0-4d53-4b73-966b-3d96c4fc33ff service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:00.398075 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ccf372c0-4d53-4b73-966b-3d96c4fc33ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:00.454144 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf372c0-4d53-4b73-966b-3d96c4fc33ff service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:00.457790 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a696c-8b10-4be5-aa6b-d787bdc3a947 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:00.458433 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a696c-8b10-4be5-aa6b-d787bdc3a947 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:00.458718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a696c-8b10-4be5-aa6b-d787bdc3a947 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:00.459022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a696c-8b10-4be5-aa6b-d787bdc3a947 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:00.461392 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-949a696c-8b10-4be5-aa6b-d787bdc3a947 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:00.476910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:00.528252 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a696c-8b10-4be5-aa6b-d787bdc3a947 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:00.529727 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a696c-8b10-4be5-aa6b-d787bdc3a947 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:00.539662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1349/2689] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:32:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:32:01.368334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:01.477788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:01.550799 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:01.659314 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:02.379740 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:02.814500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:04.854677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:05.493908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2da980-9091-44b4-b75b-04c6f94163e5 service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:05.503072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2da980-9091-44b4-b75b-04c6f94163e5 service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:05.507223 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415bbd6-77d8-46f9-8421-83b0752fe6aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:05.514153 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415bbd6-77d8-46f9-8421-83b0752fe6aa service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:05.514153 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415bbd6-77d8-46f9-8421-83b0752fe6aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:05.514153 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415bbd6-77d8-46f9-8421-83b0752fe6aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:05.514153 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-8415bbd6-77d8-46f9-8421-83b0752fe6aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:05.582758 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415bbd6-77d8-46f9-8421-83b0752fe6aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:05.584412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415bbd6-77d8-46f9-8421-83b0752fe6aa service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:05.594977 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1341/2690] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:32:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:32:06.825226 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:08.570492 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a696c-8b10-4be5-aa6b-d787bdc3a947 service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:08.580854 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a696c-8b10-4be5-aa6b-d787bdc3a947 service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:08.587152 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6cf32-330d-42c2-8195-25ecbed31772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:08.588284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6cf32-330d-42c2-8195-25ecbed31772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:08.588748 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6cf32-330d-42c2-8195-25ecbed31772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:08.589268 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6cf32-330d-42c2-8195-25ecbed31772 admin admin] PATH_INFO: `/v3/users/5c2ebc8ef0884ab1940ccaa4a33af156` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:08.590577 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6cf32-330d-42c2-8195-25ecbed31772 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c2ebc8ef0884ab1940ccaa4a33af156)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:08.592654 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6cf32-330d-42c2-8195-25ecbed31772 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:08.875498 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7ae6cf32-330d-42c2-8195-25ecbed31772 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c2ebc8ef0884ab1940ccaa4a33af156'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:08.883673 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7ae6cf32-330d-42c2-8195-25ecbed31772 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c2ebc8ef0884ab1940ccaa4a33af156'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:08.884482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7ae6cf32-330d-42c2-8195-25ecbed31772 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c2ebc8ef0884ab1940ccaa4a33af156'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:08.885207 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7ae6cf32-330d-42c2-8195-25ecbed31772 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c2ebc8ef0884ab1940ccaa4a33af156'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:08.897793 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7ae6cf32-330d-42c2-8195-25ecbed31772 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c2ebc8ef0884ab1940ccaa4a33af156'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:09.025812 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1350/2691] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:32:08 2022] DELETE /identity/v3/users/5c2ebc8ef0884ab1940ccaa4a33af156 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:32:09.033803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8ae9b-aa01-46ce-ace5-6a1536bab92f None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:09.033803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8ae9b-aa01-46ce-ace5-6a1536bab92f None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:09.033803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8ae9b-aa01-46ce-ace5-6a1536bab92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:09.053366 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-22f8ae9b-aa01-46ce-ace5-6a1536bab92f None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:32:09.055769 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-22f8ae9b-aa01-46ce-ace5-6a1536bab92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:09.062101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:09.162249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1342/2692] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:32:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:32:09.168577 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6cf32-330d-42c2-8195-25ecbed31772 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:09.170424 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-7ae6cf32-330d-42c2-8195-25ecbed31772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:09.237928 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6cf32-330d-42c2-8195-25ecbed31772 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:09.243416 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069221c-2bf7-45ac-b667-c73b59503e5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:09.244387 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069221c-2bf7-45ac-b667-c73b59503e5b service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:09.244682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069221c-2bf7-45ac-b667-c73b59503e5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:09.244969 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069221c-2bf7-45ac-b667-c73b59503e5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:09.260705 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069221c-2bf7-45ac-b667-c73b59503e5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:09.260705 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069221c-2bf7-45ac-b667-c73b59503e5b service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:09.274974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1351/2693] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:32:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:32:09.844424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8ae9b-aa01-46ce-ace5-6a1536bab92f None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:09.846038 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-22f8ae9b-aa01-46ce-ace5-6a1536bab92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:09.864630 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8ae9b-aa01-46ce-ace5-6a1536bab92f None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:09.879535 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae94687-ad98-44c2-9df0-11c68cec83bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:09.880889 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae94687-ad98-44c2-9df0-11c68cec83bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:09.881410 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae94687-ad98-44c2-9df0-11c68cec83bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:09.881985 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae94687-ad98-44c2-9df0-11c68cec83bb admin admin] PATH_INFO: `/v3/projects/3000869310b14b71ae077d3ecb73376c` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:09.883665 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae94687-ad98-44c2-9df0-11c68cec83bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3000869310b14b71ae077d3ecb73376c)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:09.885855 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae94687-ad98-44c2-9df0-11c68cec83bb admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:10.059630 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-dae94687-ad98-44c2-9df0-11c68cec83bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3000869310b14b71ae077d3ecb73376c', 'request_id': 'req-dae94687-ad98-44c2-9df0-11c68cec83bb'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:10.066185 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-dae94687-ad98-44c2-9df0-11c68cec83bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3000869310b14b71ae077d3ecb73376c', 'request_id': 'req-dae94687-ad98-44c2-9df0-11c68cec83bb'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:10.073794 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-dae94687-ad98-44c2-9df0-11c68cec83bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3000869310b14b71ae077d3ecb73376c', 'request_id': 'req-dae94687-ad98-44c2-9df0-11c68cec83bb'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:10.242342 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-dae94687-ad98-44c2-9df0-11c68cec83bb admin admin] The token cache is being invalidate because project 3000869310b14b71ae077d3ecb73376c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:32:10.242973 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-dae94687-ad98-44c2-9df0-11c68cec83bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:10.243607 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-dae94687-ad98-44c2-9df0-11c68cec83bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:10.245524 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1343/2694] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:32:09 2022] DELETE /identity/v3/projects/3000869310b14b71ae077d3ecb73376c => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:32:10.249573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069221c-2bf7-45ac-b667-c73b59503e5b service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:10.260734 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069221c-2bf7-45ac-b667-c73b59503e5b service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:10.296842 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:10.323188 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc961c2f-fa3b-42a7-8d28-41fa236b349d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:10.324072 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc961c2f-fa3b-42a7-8d28-41fa236b349d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:10.324834 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc961c2f-fa3b-42a7-8d28-41fa236b349d admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:10.325116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc961c2f-fa3b-42a7-8d28-41fa236b349d admin admin] PATH_INFO: `/v3/users/7f679565035f4f50931699ece9cb4045` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:10.326389 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc961c2f-fa3b-42a7-8d28-41fa236b349d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f679565035f4f50931699ece9cb4045)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:10.328356 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc961c2f-fa3b-42a7-8d28-41fa236b349d admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:10.629852 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-dc961c2f-fa3b-42a7-8d28-41fa236b349d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f679565035f4f50931699ece9cb4045'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:10.637663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-dc961c2f-fa3b-42a7-8d28-41fa236b349d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f679565035f4f50931699ece9cb4045'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:10.638374 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-dc961c2f-fa3b-42a7-8d28-41fa236b349d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f679565035f4f50931699ece9cb4045'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:10.639052 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-dc961c2f-fa3b-42a7-8d28-41fa236b349d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f679565035f4f50931699ece9cb4045'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:10.651464 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-dc961c2f-fa3b-42a7-8d28-41fa236b349d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f679565035f4f50931699ece9cb4045'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:10.802497 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1352/2695] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:32:10 2022] DELETE /identity/v3/users/7f679565035f4f50931699ece9cb4045 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:32:11.259565 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae94687-ad98-44c2-9df0-11c68cec83bb admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:11.261198 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-dae94687-ad98-44c2-9df0-11c68cec83bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:11.275898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae94687-ad98-44c2-9df0-11c68cec83bb admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:11.337245 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61e9c6-1e52-40ad-ad4a-f714ef3ca64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:11.338131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61e9c6-1e52-40ad-ad4a-f714ef3ca64a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:11.338448 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61e9c6-1e52-40ad-ad4a-f714ef3ca64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:11.338722 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61e9c6-1e52-40ad-ad4a-f714ef3ca64a admin admin] PATH_INFO: `/v3/users/ccb1c86f20ff4e83976cae0a8bc58936` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:11.339981 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61e9c6-1e52-40ad-ad4a-f714ef3ca64a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccb1c86f20ff4e83976cae0a8bc58936)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:11.341905 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61e9c6-1e52-40ad-ad4a-f714ef3ca64a admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:11.468690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc961c2f-fa3b-42a7-8d28-41fa236b349d admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:11.469299 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-dc961c2f-fa3b-42a7-8d28-41fa236b349d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:11.483673 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc961c2f-fa3b-42a7-8d28-41fa236b349d admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:11.489547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb037c3e-84a3-41e2-a418-dc41095edcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:11.490547 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb037c3e-84a3-41e2-a418-dc41095edcd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:11.491434 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb037c3e-84a3-41e2-a418-dc41095edcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:11.491434 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb037c3e-84a3-41e2-a418-dc41095edcd1 admin admin] PATH_INFO: `/v3/projects/187845dc71b6464c88a7f95902718ba1` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:11.493143 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb037c3e-84a3-41e2-a418-dc41095edcd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=187845dc71b6464c88a7f95902718ba1)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:11.495167 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb037c3e-84a3-41e2-a418-dc41095edcd1 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:11.638032 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:11.656017 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-bd61e9c6-1e52-40ad-ad4a-f714ef3ca64a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccb1c86f20ff4e83976cae0a8bc58936'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:11.656699 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-eb037c3e-84a3-41e2-a418-dc41095edcd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '187845dc71b6464c88a7f95902718ba1', 'request_id': 'req-eb037c3e-84a3-41e2-a418-dc41095edcd1'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:11.665205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-eb037c3e-84a3-41e2-a418-dc41095edcd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '187845dc71b6464c88a7f95902718ba1', 'request_id': 'req-eb037c3e-84a3-41e2-a418-dc41095edcd1'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:11.666318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-bd61e9c6-1e52-40ad-ad4a-f714ef3ca64a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccb1c86f20ff4e83976cae0a8bc58936'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:11.667168 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-bd61e9c6-1e52-40ad-ad4a-f714ef3ca64a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccb1c86f20ff4e83976cae0a8bc58936'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:11.667853 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-bd61e9c6-1e52-40ad-ad4a-f714ef3ca64a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccb1c86f20ff4e83976cae0a8bc58936'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:11.680333 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-bd61e9c6-1e52-40ad-ad4a-f714ef3ca64a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccb1c86f20ff4e83976cae0a8bc58936'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:11.762335 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-eb037c3e-84a3-41e2-a418-dc41095edcd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '187845dc71b6464c88a7f95902718ba1', 'request_id': 'req-eb037c3e-84a3-41e2-a418-dc41095edcd1'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:11.773894 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-eb037c3e-84a3-41e2-a418-dc41095edcd1 admin admin] The token cache is being invalidate because project 187845dc71b6464c88a7f95902718ba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:32:11.774357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-eb037c3e-84a3-41e2-a418-dc41095edcd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:11.775408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-eb037c3e-84a3-41e2-a418-dc41095edcd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:11.777046 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1353/2696] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:32:11 2022] DELETE /identity/v3/projects/187845dc71b6464c88a7f95902718ba1 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:32:11.778085 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb037c3e-84a3-41e2-a418-dc41095edcd1 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:11.779451 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-eb037c3e-84a3-41e2-a418-dc41095edcd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:11.842941 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb037c3e-84a3-41e2-a418-dc41095edcd1 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:11.847701 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245da57-ac60-4987-8074-5429cadb872b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:11.848580 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245da57-ac60-4987-8074-5429cadb872b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:11.848869 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245da57-ac60-4987-8074-5429cadb872b admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:11.849153 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245da57-ac60-4987-8074-5429cadb872b admin admin] PATH_INFO: `/v3/users/65220457c29c46358fee054fb9366406` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:11.850370 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245da57-ac60-4987-8074-5429cadb872b admin admin] RBAC: Authorizing `identity:delete_user(user_id=65220457c29c46358fee054fb9366406)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:11.852873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245da57-ac60-4987-8074-5429cadb872b admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:11.919045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1344/2697] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:32:11 2022] DELETE /identity/v3/users/ccb1c86f20ff4e83976cae0a8bc58936 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:32:11.919045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2a369-a6e1-4638-b7f7-d5136b84e8f5 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:11.919045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2a369-a6e1-4638-b7f7-d5136b84e8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:11.919045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2a369-a6e1-4638-b7f7-d5136b84e8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:11.935724 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-65d2a369-a6e1-4638-b7f7-d5136b84e8f5 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:32:11.936904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-65d2a369-a6e1-4638-b7f7-d5136b84e8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:11.996365 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1345/2698] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:32:11.997717 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e673f0b-bf34-4c03-a93d-7d640d1205c0 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:11.998011 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e673f0b-bf34-4c03-a93d-7d640d1205c0 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:11.998294 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e673f0b-bf34-4c03-a93d-7d640d1205c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:12.014893 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-5e673f0b-bf34-4c03-a93d-7d640d1205c0 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:32:12.016135 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-5e673f0b-bf34-4c03-a93d-7d640d1205c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:12.027411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1346/2699] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:32:12.027411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561b1b5-a84a-46be-a8c3-ebf06b737425 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:12.027411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561b1b5-a84a-46be-a8c3-ebf06b737425 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:12.027411 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561b1b5-a84a-46be-a8c3-ebf06b737425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:12.044582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-6561b1b5-a84a-46be-a8c3-ebf06b737425 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:32:12.045761 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-6561b1b5-a84a-46be-a8c3-ebf06b737425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:12.053646 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1347/2700] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:32:12.054439 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561b1b5-a84a-46be-a8c3-ebf06b737425 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:12.055657 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-6561b1b5-a84a-46be-a8c3-ebf06b737425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:12.109461 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561b1b5-a84a-46be-a8c3-ebf06b737425 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:12.113064 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b7cc0-39ca-4349-95cc-129c138f3b9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:12.114061 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b7cc0-39ca-4349-95cc-129c138f3b9d service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:12.114348 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b7cc0-39ca-4349-95cc-129c138f3b9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:12.114628 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b7cc0-39ca-4349-95cc-129c138f3b9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:12.124181 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b7cc0-39ca-4349-95cc-129c138f3b9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:12.125729 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b7cc0-39ca-4349-95cc-129c138f3b9d service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:12.135681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1348/2701] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:32:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:32:12.136459 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b7cc0-39ca-4349-95cc-129c138f3b9d service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:12.145912 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b7cc0-39ca-4349-95cc-129c138f3b9d service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:12.151208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f247ca-c1dc-4511-9cda-5d6f40a9d52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:12.152342 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f247ca-c1dc-4511-9cda-5d6f40a9d52b admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:12.152342 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f247ca-c1dc-4511-9cda-5d6f40a9d52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:12.152625 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f247ca-c1dc-4511-9cda-5d6f40a9d52b admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:12.153428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f247ca-c1dc-4511-9cda-5d6f40a9d52b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:32:12.153809 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f247ca-c1dc-4511-9cda-5d6f40a9d52b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:12.155672 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f247ca-c1dc-4511-9cda-5d6f40a9d52b admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:12.170878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1349/2702] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:32:12.172017 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f247ca-c1dc-4511-9cda-5d6f40a9d52b admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:12.182832 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f247ca-c1dc-4511-9cda-5d6f40a9d52b admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:12.187757 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0a99d-e700-477f-8828-4df9276c2bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:12.193692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0a99d-e700-477f-8828-4df9276c2bf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:12.193692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0a99d-e700-477f-8828-4df9276c2bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:12.193692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0a99d-e700-477f-8828-4df9276c2bf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:12.193692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f0a99d-e700-477f-8828-4df9276c2bf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:12.193692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f0a99d-e700-477f-8828-4df9276c2bf2 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:12.228663 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-5245da57-ac60-4987-8074-5429cadb872b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65220457c29c46358fee054fb9366406'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:12.235525 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-5245da57-ac60-4987-8074-5429cadb872b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65220457c29c46358fee054fb9366406'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:12.236186 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-5245da57-ac60-4987-8074-5429cadb872b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65220457c29c46358fee054fb9366406'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:12.236791 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-5245da57-ac60-4987-8074-5429cadb872b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65220457c29c46358fee054fb9366406'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:12.249684 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-5245da57-ac60-4987-8074-5429cadb872b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65220457c29c46358fee054fb9366406'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:12.328661 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1350/2703] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:32:12 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:32:12.332660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0a99d-e700-477f-8828-4df9276c2bf2 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:12.336447 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-81f0a99d-e700-477f-8828-4df9276c2bf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:12.378599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1354/2704] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:32:11 2022] DELETE /identity/v3/users/65220457c29c46358fee054fb9366406 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:32:12.410586 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0a99d-e700-477f-8828-4df9276c2bf2 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:12.481219 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff784f4f-749a-4041-9006-e503328a87c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:12.482065 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff784f4f-749a-4041-9006-e503328a87c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:12.482456 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff784f4f-749a-4041-9006-e503328a87c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:12.482831 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff784f4f-749a-4041-9006-e503328a87c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:12.483658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff784f4f-749a-4041-9006-e503328a87c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:12.485874 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff784f4f-749a-4041-9006-e503328a87c8 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:12.573867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1351/2705] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:32:12 2022] POST /identity/v3/users => generated 292 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:32:12.577771 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245da57-ac60-4987-8074-5429cadb872b admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:12.587873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245da57-ac60-4987-8074-5429cadb872b admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:12.592454 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600d851-3291-4230-bf26-33ddeda0a9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:12.593115 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600d851-3291-4230-bf26-33ddeda0a9c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:12.593404 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600d851-3291-4230-bf26-33ddeda0a9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:12.593687 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600d851-3291-4230-bf26-33ddeda0a9c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:12.594415 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600d851-3291-4230-bf26-33ddeda0a9c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:12.596055 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600d851-3291-4230-bf26-33ddeda0a9c3 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:12.606270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1355/2706] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:32:12.610233 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff784f4f-749a-4041-9006-e503328a87c8 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:12.620881 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff784f4f-749a-4041-9006-e503328a87c8 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:12.625458 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c88303-1623-4355-9cf3-83400f27ee58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:12.626111 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c88303-1623-4355-9cf3-83400f27ee58 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:12.626398 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c88303-1623-4355-9cf3-83400f27ee58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:12.626698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c88303-1623-4355-9cf3-83400f27ee58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:12.627460 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c88303-1623-4355-9cf3-83400f27ee58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:12.629088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c88303-1623-4355-9cf3-83400f27ee58 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:12.639018 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1352/2707] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:32:12.644843 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600d851-3291-4230-bf26-33ddeda0a9c3 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:12.654872 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600d851-3291-4230-bf26-33ddeda0a9c3 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:12.659472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5cbfd0-d609-45dc-bb7d-f2b20af0cc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:12.660376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5cbfd0-d609-45dc-bb7d-f2b20af0cc3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:12.660664 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5cbfd0-d609-45dc-bb7d-f2b20af0cc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:12.660943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5cbfd0-d609-45dc-bb7d-f2b20af0cc3a admin admin] PATH_INFO: `/v3/projects/e06cabc6fd104c7cbd2758700a0d29a8/users/ee3ad4a33625438fb5c0ae12314689d6/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:12.675934 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5cbfd0-d609-45dc-bb7d-f2b20af0cc3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e06cabc6fd104c7cbd2758700a0d29a8, user_id=ee3ad4a33625438fb5c0ae12314689d6, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:12.678523 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5cbfd0-d609-45dc-bb7d-f2b20af0cc3a admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:12.693207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c88303-1623-4355-9cf3-83400f27ee58 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:12.695068 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-72c88303-1623-4355-9cf3-83400f27ee58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:12.708843 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c88303-1623-4355-9cf3-83400f27ee58 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:12.715182 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f3d03-61a5-47ee-b61d-9dc26e438356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:12.716061 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f3d03-61a5-47ee-b61d-9dc26e438356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:12.722125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f3d03-61a5-47ee-b61d-9dc26e438356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:12.722125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f3d03-61a5-47ee-b61d-9dc26e438356 admin admin] PATH_INFO: `/v3/projects/3ee3f1c33377446489b75e9faaa5654b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:12.722125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f3d03-61a5-47ee-b61d-9dc26e438356 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ee3f1c33377446489b75e9faaa5654b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:12.722125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f3d03-61a5-47ee-b61d-9dc26e438356 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:12.820021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1356/2708] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:32:12 2022] PUT /identity/v3/projects/e06cabc6fd104c7cbd2758700a0d29a8/users/ee3ad4a33625438fb5c0ae12314689d6/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:32:12.821054 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5cbfd0-d609-45dc-bb7d-f2b20af0cc3a admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:12.822678 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-9c5cbfd0-d609-45dc-bb7d-f2b20af0cc3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:12.901859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-534f3d03-61a5-47ee-b61d-9dc26e438356 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ee3f1c33377446489b75e9faaa5654b', 'request_id': 'req-534f3d03-61a5-47ee-b61d-9dc26e438356'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:12.905134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5cbfd0-d609-45dc-bb7d-f2b20af0cc3a admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:12.908677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-534f3d03-61a5-47ee-b61d-9dc26e438356 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ee3f1c33377446489b75e9faaa5654b', 'request_id': 'req-534f3d03-61a5-47ee-b61d-9dc26e438356'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:12.914847 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-534f3d03-61a5-47ee-b61d-9dc26e438356 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ee3f1c33377446489b75e9faaa5654b', 'request_id': 'req-534f3d03-61a5-47ee-b61d-9dc26e438356'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:12.961287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fcd27-666f-47cd-b004-2e363da6c587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:12.962057 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fcd27-666f-47cd-b004-2e363da6c587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:12.962383 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fcd27-666f-47cd-b004-2e363da6c587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:12.962714 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fcd27-666f-47cd-b004-2e363da6c587 admin admin] PATH_INFO: `/v3/projects/9541069573d04541b3d567f3c06674df` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:12.967745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361fcd27-666f-47cd-b004-2e363da6c587 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9541069573d04541b3d567f3c06674df)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:12.976751 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361fcd27-666f-47cd-b004-2e363da6c587 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:13.012500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-534f3d03-61a5-47ee-b61d-9dc26e438356 admin admin] The token cache is being invalidate because project 3ee3f1c33377446489b75e9faaa5654b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:32:13.012795 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-534f3d03-61a5-47ee-b61d-9dc26e438356 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:13.013439 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-534f3d03-61a5-47ee-b61d-9dc26e438356 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:13.015048 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1353/2709] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:32:12 2022] DELETE /identity/v3/projects/3ee3f1c33377446489b75e9faaa5654b => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:32:13.016292 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f184193-6058-4241-a1c4-33f92c195311 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:13.016582 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f184193-6058-4241-a1c4-33f92c195311 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:13.016866 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f184193-6058-4241-a1c4-33f92c195311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:13.034207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-6f184193-6058-4241-a1c4-33f92c195311 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:32:13.035556 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-6f184193-6058-4241-a1c4-33f92c195311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:13.043970 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1354/2710] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:32:13.044580 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f184193-6058-4241-a1c4-33f92c195311 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:13.045869 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-6f184193-6058-4241-a1c4-33f92c195311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:13.057623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f184193-6058-4241-a1c4-33f92c195311 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:13.062236 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4530c-db85-4bae-9bf5-275667ff9059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:13.063120 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4530c-db85-4bae-9bf5-275667ff9059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:13.063413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4530c-db85-4bae-9bf5-275667ff9059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:13.063696 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4530c-db85-4bae-9bf5-275667ff9059 admin admin] PATH_INFO: `/v3/users/ec8902d03f434aa980e66acd0d159ddb` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:13.064894 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f4530c-db85-4bae-9bf5-275667ff9059 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec8902d03f434aa980e66acd0d159ddb)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:13.066814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f4530c-db85-4bae-9bf5-275667ff9059 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:13.116934 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:13.122726 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-361fcd27-666f-47cd-b004-2e363da6c587 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9541069573d04541b3d567f3c06674df', 'request_id': 'req-361fcd27-666f-47cd-b004-2e363da6c587'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:13.128869 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-361fcd27-666f-47cd-b004-2e363da6c587 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9541069573d04541b3d567f3c06674df', 'request_id': 'req-361fcd27-666f-47cd-b004-2e363da6c587'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:13.201882 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-361fcd27-666f-47cd-b004-2e363da6c587 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9541069573d04541b3d567f3c06674df', 'request_id': 'req-361fcd27-666f-47cd-b004-2e363da6c587'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:13.214386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-361fcd27-666f-47cd-b004-2e363da6c587 admin admin] The token cache is being invalidate because project 9541069573d04541b3d567f3c06674df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:32:13.214386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-361fcd27-666f-47cd-b004-2e363da6c587 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:13.218095 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-361fcd27-666f-47cd-b004-2e363da6c587 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:13.218095 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1357/2711] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:32:12 2022] DELETE /identity/v3/projects/9541069573d04541b3d567f3c06674df => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:32:13.218095 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fcd27-666f-47cd-b004-2e363da6c587 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:13.219148 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-361fcd27-666f-47cd-b004-2e363da6c587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:13.276534 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fcd27-666f-47cd-b004-2e363da6c587 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:13.280319 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a76bf99-c724-49a8-b3bf-0e31f9e42b71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:13.284058 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a76bf99-c724-49a8-b3bf-0e31f9e42b71 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:13.284058 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a76bf99-c724-49a8-b3bf-0e31f9e42b71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:13.284058 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a76bf99-c724-49a8-b3bf-0e31f9e42b71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:13.284058 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-6a76bf99-c724-49a8-b3bf-0e31f9e42b71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:13.289991 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b9f4530c-db85-4bae-9bf5-275667ff9059 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec8902d03f434aa980e66acd0d159ddb'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:13.296928 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b9f4530c-db85-4bae-9bf5-275667ff9059 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec8902d03f434aa980e66acd0d159ddb'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:13.297695 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b9f4530c-db85-4bae-9bf5-275667ff9059 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec8902d03f434aa980e66acd0d159ddb'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:13.298304 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b9f4530c-db85-4bae-9bf5-275667ff9059 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec8902d03f434aa980e66acd0d159ddb'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:13.311462 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-b9f4530c-db85-4bae-9bf5-275667ff9059 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec8902d03f434aa980e66acd0d159ddb'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:13.346981 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a76bf99-c724-49a8-b3bf-0e31f9e42b71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:13.348477 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a76bf99-c724-49a8-b3bf-0e31f9e42b71 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:13.358439 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1358/2712] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:32:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:32:13.359344 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a76bf99-c724-49a8-b3bf-0e31f9e42b71 service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:13.360734 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-6a76bf99-c724-49a8-b3bf-0e31f9e42b71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:13.373340 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a76bf99-c724-49a8-b3bf-0e31f9e42b71 service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:13.378637 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa4e046-e07d-480f-959e-1f9be5cae5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:13.379571 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4e046-e07d-480f-959e-1f9be5cae5d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:13.379856 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4e046-e07d-480f-959e-1f9be5cae5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:13.380136 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4e046-e07d-480f-959e-1f9be5cae5d5 admin admin] PATH_INFO: `/v3/users/255c8ecc45724283a4977ffcceca70f7` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:13.381355 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa4e046-e07d-480f-959e-1f9be5cae5d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=255c8ecc45724283a4977ffcceca70f7)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:13.383947 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa4e046-e07d-480f-959e-1f9be5cae5d5 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:13.390323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1355/2713] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:32:13 2022] DELETE /identity/v3/users/ec8902d03f434aa980e66acd0d159ddb => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:32:13.615405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7aa4e046-e07d-480f-959e-1f9be5cae5d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '255c8ecc45724283a4977ffcceca70f7'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:13.622550 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7aa4e046-e07d-480f-959e-1f9be5cae5d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '255c8ecc45724283a4977ffcceca70f7'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:13.623232 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7aa4e046-e07d-480f-959e-1f9be5cae5d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '255c8ecc45724283a4977ffcceca70f7'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:13.623858 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7aa4e046-e07d-480f-959e-1f9be5cae5d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '255c8ecc45724283a4977ffcceca70f7'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:13.635913 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-7aa4e046-e07d-480f-959e-1f9be5cae5d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '255c8ecc45724283a4977ffcceca70f7'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:13.695543 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4530c-db85-4bae-9bf5-275667ff9059 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:13.696987 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b9f4530c-db85-4bae-9bf5-275667ff9059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:13.733940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1359/2714] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:32:13 2022] DELETE /identity/v3/users/255c8ecc45724283a4977ffcceca70f7 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:32:13.779303 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4530c-db85-4bae-9bf5-275667ff9059 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:13.847820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210598ab-5a90-460c-922c-3ec3d5ed4c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:13.848688 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210598ab-5a90-460c-922c-3ec3d5ed4c93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:13.848973 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210598ab-5a90-460c-922c-3ec3d5ed4c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:13.849249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210598ab-5a90-460c-922c-3ec3d5ed4c93 admin admin] PATH_INFO: `/v3/projects/ef7b8d3e156e464fa3330d4376b39310` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:13.850439 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210598ab-5a90-460c-922c-3ec3d5ed4c93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef7b8d3e156e464fa3330d4376b39310)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:13.853023 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210598ab-5a90-460c-922c-3ec3d5ed4c93 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:14.014990 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-210598ab-5a90-460c-922c-3ec3d5ed4c93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef7b8d3e156e464fa3330d4376b39310', 'request_id': 'req-210598ab-5a90-460c-922c-3ec3d5ed4c93'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:14.026619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-210598ab-5a90-460c-922c-3ec3d5ed4c93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef7b8d3e156e464fa3330d4376b39310', 'request_id': 'req-210598ab-5a90-460c-922c-3ec3d5ed4c93'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:14.032777 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-210598ab-5a90-460c-922c-3ec3d5ed4c93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef7b8d3e156e464fa3330d4376b39310', 'request_id': 'req-210598ab-5a90-460c-922c-3ec3d5ed4c93'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:14.088031 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-210598ab-5a90-460c-922c-3ec3d5ed4c93 admin admin] The token cache is being invalidate because project ef7b8d3e156e464fa3330d4376b39310 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:32:14.088445 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-210598ab-5a90-460c-922c-3ec3d5ed4c93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:14.089224 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-210598ab-5a90-460c-922c-3ec3d5ed4c93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:14.090954 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1356/2715] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:32:13 2022] DELETE /identity/v3/projects/ef7b8d3e156e464fa3330d4376b39310 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:32:14.336522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa4e046-e07d-480f-959e-1f9be5cae5d5 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:14.338130 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-7aa4e046-e07d-480f-959e-1f9be5cae5d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:14.352533 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa4e046-e07d-480f-959e-1f9be5cae5d5 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:14.411160 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347f27b-823a-477f-8353-6d05474d189e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:14.412029 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347f27b-823a-477f-8353-6d05474d189e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:14.412318 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347f27b-823a-477f-8353-6d05474d189e admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:14.412626 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347f27b-823a-477f-8353-6d05474d189e admin admin] PATH_INFO: `/v3/projects/db063cba941b47faace2dfc86e9885b5` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:14.414420 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347f27b-823a-477f-8353-6d05474d189e admin admin] RBAC: Authorizing `identity:delete_project(project_id=db063cba941b47faace2dfc86e9885b5)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:14.416416 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347f27b-823a-477f-8353-6d05474d189e admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:14.551382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0347f27b-823a-477f-8353-6d05474d189e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db063cba941b47faace2dfc86e9885b5', 'request_id': 'req-0347f27b-823a-477f-8353-6d05474d189e'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:14.557618 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0347f27b-823a-477f-8353-6d05474d189e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db063cba941b47faace2dfc86e9885b5', 'request_id': 'req-0347f27b-823a-477f-8353-6d05474d189e'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:14.636204 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0347f27b-823a-477f-8353-6d05474d189e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db063cba941b47faace2dfc86e9885b5', 'request_id': 'req-0347f27b-823a-477f-8353-6d05474d189e'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:14.650158 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0347f27b-823a-477f-8353-6d05474d189e admin admin] The token cache is being invalidate because project db063cba941b47faace2dfc86e9885b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:32:14.651952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0347f27b-823a-477f-8353-6d05474d189e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:14.651952 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0347f27b-823a-477f-8353-6d05474d189e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:14.654016 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:14.656985 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1360/2716] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:32:14 2022] DELETE /identity/v3/projects/db063cba941b47faace2dfc86e9885b5 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:32:14.657329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210598ab-5a90-460c-922c-3ec3d5ed4c93 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:14.658785 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-210598ab-5a90-460c-922c-3ec3d5ed4c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:14.671664 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210598ab-5a90-460c-922c-3ec3d5ed4c93 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:14.732637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d620e16-c390-4a65-b50c-d88d168584b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:14.733656 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d620e16-c390-4a65-b50c-d88d168584b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:14.733949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d620e16-c390-4a65-b50c-d88d168584b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:14.734231 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d620e16-c390-4a65-b50c-d88d168584b0 admin admin] PATH_INFO: `/v3/users/403f03b9ec044e8781881fe24b2c84fe` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:14.736084 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629441a3-8d97-4537-921e-185b9a8e3888 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:14.736084 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629441a3-8d97-4537-921e-185b9a8e3888 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:14.736084 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629441a3-8d97-4537-921e-185b9a8e3888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:14.736385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d620e16-c390-4a65-b50c-d88d168584b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=403f03b9ec044e8781881fe24b2c84fe)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:14.737549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d620e16-c390-4a65-b50c-d88d168584b0 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:14.766320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-629441a3-8d97-4537-921e-185b9a8e3888 None None] MFA Rules not processed for user `ee3ad4a33625438fb5c0ae12314689d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:32:14.767605 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-629441a3-8d97-4537-921e-185b9a8e3888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:14.839529 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1361/2717] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:32:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:32:14.847361 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629441a3-8d97-4537-921e-185b9a8e3888 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:14.848763 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-629441a3-8d97-4537-921e-185b9a8e3888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:14.906272 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629441a3-8d97-4537-921e-185b9a8e3888 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:14.909931 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9b9ba-b269-4197-b551-66f216e639c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:14.910587 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9b9ba-b269-4197-b551-66f216e639c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:14.910873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9b9ba-b269-4197-b551-66f216e639c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:14.911180 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9b9ba-b269-4197-b551-66f216e639c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:14.920200 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb9b9ba-b269-4197-b551-66f216e639c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:14.921656 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb9b9ba-b269-4197-b551-66f216e639c2 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:14.932079 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1362/2718] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:32:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:32:14.982683 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-4d620e16-c390-4a65-b50c-d88d168584b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '403f03b9ec044e8781881fe24b2c84fe'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:14.991281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-4d620e16-c390-4a65-b50c-d88d168584b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '403f03b9ec044e8781881fe24b2c84fe'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:14.992988 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-4d620e16-c390-4a65-b50c-d88d168584b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '403f03b9ec044e8781881fe24b2c84fe'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:14.993743 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-4d620e16-c390-4a65-b50c-d88d168584b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '403f03b9ec044e8781881fe24b2c84fe'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:15.009662 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-4d620e16-c390-4a65-b50c-d88d168584b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '403f03b9ec044e8781881fe24b2c84fe'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:15.077639 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1357/2719] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:32:14 2022] DELETE /identity/v3/users/403f03b9ec044e8781881fe24b2c84fe => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:32:15.557813 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9b9ba-b269-4197-b551-66f216e639c2 service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:15.559509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-deb9b9ba-b269-4197-b551-66f216e639c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:15.575926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9b9ba-b269-4197-b551-66f216e639c2 service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:15.631807 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553cd87b-b547-4040-9797-4b5718937312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:15.633170 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553cd87b-b547-4040-9797-4b5718937312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:15.633454 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553cd87b-b547-4040-9797-4b5718937312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:15.633731 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553cd87b-b547-4040-9797-4b5718937312 admin admin] PATH_INFO: `/v3/projects/2938e65d9cb34242a8bbac7c16ba6d6b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:15.634950 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553cd87b-b547-4040-9797-4b5718937312 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2938e65d9cb34242a8bbac7c16ba6d6b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:15.636865 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553cd87b-b547-4040-9797-4b5718937312 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:15.722448 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:15.780412 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-553cd87b-b547-4040-9797-4b5718937312 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2938e65d9cb34242a8bbac7c16ba6d6b', 'request_id': 'req-553cd87b-b547-4040-9797-4b5718937312'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:15.786528 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-553cd87b-b547-4040-9797-4b5718937312 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2938e65d9cb34242a8bbac7c16ba6d6b', 'request_id': 'req-553cd87b-b547-4040-9797-4b5718937312'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:15.887951 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-553cd87b-b547-4040-9797-4b5718937312 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2938e65d9cb34242a8bbac7c16ba6d6b', 'request_id': 'req-553cd87b-b547-4040-9797-4b5718937312'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:15.900167 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-553cd87b-b547-4040-9797-4b5718937312 admin admin] The token cache is being invalidate because project 2938e65d9cb34242a8bbac7c16ba6d6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:32:15.900452 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-553cd87b-b547-4040-9797-4b5718937312 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:15.901146 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-553cd87b-b547-4040-9797-4b5718937312 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:15.902945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1363/2720] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:32:15 2022] DELETE /identity/v3/projects/2938e65d9cb34242a8bbac7c16ba6d6b => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:32:15.908735 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de55ac-8045-450c-aa98-c54541e64ac9 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:15.909260 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de55ac-8045-450c-aa98-c54541e64ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:15.909364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de55ac-8045-450c-aa98-c54541e64ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:15.928407 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-42de55ac-8045-450c-aa98-c54541e64ac9 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:32:15.930164 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-42de55ac-8045-450c-aa98-c54541e64ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:15.993658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1358/2721] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:32:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:32:16.003001 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45db53-ec34-4c24-9ad9-a50d77a6c0e9 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:16.003297 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45db53-ec34-4c24-9ad9-a50d77a6c0e9 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:16.003579 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45db53-ec34-4c24-9ad9-a50d77a6c0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:16.022799 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-6f45db53-ec34-4c24-9ad9-a50d77a6c0e9 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:32:16.024139 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-6f45db53-ec34-4c24-9ad9-a50d77a6c0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:16.032167 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1364/2722] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:32:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:32:16.036669 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de55ac-8045-450c-aa98-c54541e64ac9 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:16.046471 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de55ac-8045-450c-aa98-c54541e64ac9 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:16.051326 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab420c91-06df-4ac0-821a-725e290307e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:16.052116 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab420c91-06df-4ac0-821a-725e290307e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:16.052400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab420c91-06df-4ac0-821a-725e290307e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:16.052678 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab420c91-06df-4ac0-821a-725e290307e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:16.053401 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab420c91-06df-4ac0-821a-725e290307e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:32:16.053709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab420c91-06df-4ac0-821a-725e290307e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:16.055898 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab420c91-06df-4ac0-821a-725e290307e8 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:16.068279 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1359/2723] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:32:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:32:16.073406 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45db53-ec34-4c24-9ad9-a50d77a6c0e9 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:16.082660 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45db53-ec34-4c24-9ad9-a50d77a6c0e9 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:16.087442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297b055-3495-405b-aa25-bc597815c6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:16.088098 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297b055-3495-405b-aa25-bc597815c6a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:16.088384 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297b055-3495-405b-aa25-bc597815c6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:16.088662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297b055-3495-405b-aa25-bc597815c6a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:16.089349 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5297b055-3495-405b-aa25-bc597815c6a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:16.091289 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5297b055-3495-405b-aa25-bc597815c6a2 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:16.169771 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1365/2724] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:32:16 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:32:16.174033 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab420c91-06df-4ac0-821a-725e290307e8 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:16.184926 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab420c91-06df-4ac0-821a-725e290307e8 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:16.240648 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa26446-63ad-4ec6-a6f1-c86859fe83ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:16.241321 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa26446-63ad-4ec6-a6f1-c86859fe83ef admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:16.242212 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa26446-63ad-4ec6-a6f1-c86859fe83ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:16.242499 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa26446-63ad-4ec6-a6f1-c86859fe83ef admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:16.243235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa26446-63ad-4ec6-a6f1-c86859fe83ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:16.245160 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa26446-63ad-4ec6-a6f1-c86859fe83ef admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:16.327617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1360/2725] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:32:16 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:32:16.331869 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297b055-3495-405b-aa25-bc597815c6a2 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:16.342253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297b055-3495-405b-aa25-bc597815c6a2 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:16.346966 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdbddd-4633-4abc-a6c6-e1320679900c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:16.347767 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdbddd-4633-4abc-a6c6-e1320679900c admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:16.348055 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdbddd-4633-4abc-a6c6-e1320679900c admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:16.348337 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdbddd-4633-4abc-a6c6-e1320679900c admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:16.349133 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cdbddd-4633-4abc-a6c6-e1320679900c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:16.351638 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cdbddd-4633-4abc-a6c6-e1320679900c admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:16.362478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1366/2726] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:32:16.367267 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa26446-63ad-4ec6-a6f1-c86859fe83ef admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:16.377664 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa26446-63ad-4ec6-a6f1-c86859fe83ef admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:16.384233 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c2cc4-7652-4f4e-aa8f-03be93767e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:16.384918 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c2cc4-7652-4f4e-aa8f-03be93767e5f admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:16.385676 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c2cc4-7652-4f4e-aa8f-03be93767e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:16.385676 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c2cc4-7652-4f4e-aa8f-03be93767e5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:16.386294 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c2cc4-7652-4f4e-aa8f-03be93767e5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:16.388013 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c2cc4-7652-4f4e-aa8f-03be93767e5f admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:16.399539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1361/2727] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:32:16.403713 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdbddd-4633-4abc-a6c6-e1320679900c admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:16.415921 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdbddd-4633-4abc-a6c6-e1320679900c admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:16.421575 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7ea4a-39b9-4906-9efb-b840371f31d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:16.422658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7ea4a-39b9-4906-9efb-b840371f31d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:16.422974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7ea4a-39b9-4906-9efb-b840371f31d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:16.423305 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7ea4a-39b9-4906-9efb-b840371f31d7 admin admin] PATH_INFO: `/v3/projects/a1eb2755befb44639cd693ab1a15c1ec/users/471a3a479300439aacd85114428663a2/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:16.443491 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7ea4a-39b9-4906-9efb-b840371f31d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1eb2755befb44639cd693ab1a15c1ec, user_id=471a3a479300439aacd85114428663a2, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:16.446833 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7ea4a-39b9-4906-9efb-b840371f31d7 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:16.529111 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1367/2728] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:32:16 2022] PUT /identity/v3/projects/a1eb2755befb44639cd693ab1a15c1ec/users/471a3a479300439aacd85114428663a2/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:32:16.540269 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd0808-476c-4ad3-b9f2-d5b6fcc006e1 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:16.540439 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd0808-476c-4ad3-b9f2-d5b6fcc006e1 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:16.540718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd0808-476c-4ad3-b9f2-d5b6fcc006e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:16.568925 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-93cd0808-476c-4ad3-b9f2-d5b6fcc006e1 None None] MFA Rules not processed for user `471a3a479300439aacd85114428663a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:32:16.570814 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-93cd0808-476c-4ad3-b9f2-d5b6fcc006e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:16.638477 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1362/2729] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:32:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:32:16.651448 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7ea4a-39b9-4906-9efb-b840371f31d7 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:16.653247 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-e0f7ea4a-39b9-4906-9efb-b840371f31d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:16.710848 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:16.712390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7ea4a-39b9-4906-9efb-b840371f31d7 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:16.716209 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2c268-b9f7-4480-bee4-654ca1ed7d71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:16.716862 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2c268-b9f7-4480-bee4-654ca1ed7d71 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:16.717150 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2c268-b9f7-4480-bee4-654ca1ed7d71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:16.717434 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2c268-b9f7-4480-bee4-654ca1ed7d71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:16.726691 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2c268-b9f7-4480-bee4-654ca1ed7d71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:16.728060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2c268-b9f7-4480-bee4-654ca1ed7d71 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:16.737994 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1368/2730] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:32:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:32:16.758632 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd0808-476c-4ad3-b9f2-d5b6fcc006e1 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:16.816949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c5ee5-1c85-446e-9e92-3fa055711914 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:16.817237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c5ee5-1c85-446e-9e92-3fa055711914 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:16.817533 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c5ee5-1c85-446e-9e92-3fa055711914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:16.828579 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd0808-476c-4ad3-b9f2-d5b6fcc006e1 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:16.833246 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1756c-6508-4b7e-b27b-bea3ce7c71f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:16.834201 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1756c-6508-4b7e-b27b-bea3ce7c71f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:16.834417 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1756c-6508-4b7e-b27b-bea3ce7c71f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:16.834699 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1756c-6508-4b7e-b27b-bea3ce7c71f3 admin admin] PATH_INFO: `/v3/users/471a3a479300439aacd85114428663a2` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:16.835213 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-ee7c5ee5-1c85-446e-9e92-3fa055711914 None None] MFA Rules not processed for user `ee3ad4a33625438fb5c0ae12314689d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:32:16.836056 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f1756c-6508-4b7e-b27b-bea3ce7c71f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=471a3a479300439aacd85114428663a2)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:16.836456 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ee7c5ee5-1c85-446e-9e92-3fa055711914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:16.838006 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f1756c-6508-4b7e-b27b-bea3ce7c71f3 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:16.893008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1369/2731] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:32:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:32:17.062236 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-24f1756c-6508-4b7e-b27b-bea3ce7c71f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '471a3a479300439aacd85114428663a2'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:17.069246 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-24f1756c-6508-4b7e-b27b-bea3ce7c71f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '471a3a479300439aacd85114428663a2'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:17.069945 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-24f1756c-6508-4b7e-b27b-bea3ce7c71f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '471a3a479300439aacd85114428663a2'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:17.070579 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-24f1756c-6508-4b7e-b27b-bea3ce7c71f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '471a3a479300439aacd85114428663a2'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:17.082554 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-24f1756c-6508-4b7e-b27b-bea3ce7c71f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '471a3a479300439aacd85114428663a2'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:17.168753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1363/2732] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:32:16 2022] DELETE /identity/v3/users/471a3a479300439aacd85114428663a2 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:32:17.173519 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ee225-5d66-4c4b-a170-89b499f2d78f None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:17.173819 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ee225-5d66-4c4b-a170-89b499f2d78f None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:17.174108 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ee225-5d66-4c4b-a170-89b499f2d78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:17.191860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-764ee225-5d66-4c4b-a170-89b499f2d78f None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:32:17.193066 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-764ee225-5d66-4c4b-a170-89b499f2d78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:17.254040 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1370/2733] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:32:17.264192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1756c-6508-4b7e-b27b-bea3ce7c71f3 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:17.265623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-24f1756c-6508-4b7e-b27b-bea3ce7c71f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:17.327566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1756c-6508-4b7e-b27b-bea3ce7c71f3 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:17.331436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c6f77-b840-4362-8107-687c5439f4df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:17.332298 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c6f77-b840-4362-8107-687c5439f4df service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:17.332649 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c6f77-b840-4362-8107-687c5439f4df service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:17.332942 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c6f77-b840-4362-8107-687c5439f4df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:17.342072 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691c6f77-b840-4362-8107-687c5439f4df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:17.343557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691c6f77-b840-4362-8107-687c5439f4df service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:17.353198 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1364/2734] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:32:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:32:17.379380 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:17.911859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ee225-5d66-4c4b-a170-89b499f2d78f None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:17.913877 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-764ee225-5d66-4c4b-a170-89b499f2d78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:17.926129 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ee225-5d66-4c4b-a170-89b499f2d78f None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:17.931511 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36453366-f052-4dbb-a766-e4ca5ca58591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:17.932449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36453366-f052-4dbb-a766-e4ca5ca58591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:17.932805 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36453366-f052-4dbb-a766-e4ca5ca58591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:17.933175 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36453366-f052-4dbb-a766-e4ca5ca58591 admin admin] PATH_INFO: `/v3/projects/a1eb2755befb44639cd693ab1a15c1ec` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:17.934443 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36453366-f052-4dbb-a766-e4ca5ca58591 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1eb2755befb44639cd693ab1a15c1ec)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:17.936792 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36453366-f052-4dbb-a766-e4ca5ca58591 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:18.062020 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-36453366-f052-4dbb-a766-e4ca5ca58591 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1eb2755befb44639cd693ab1a15c1ec', 'request_id': 'req-36453366-f052-4dbb-a766-e4ca5ca58591'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:18.069418 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-36453366-f052-4dbb-a766-e4ca5ca58591 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1eb2755befb44639cd693ab1a15c1ec', 'request_id': 'req-36453366-f052-4dbb-a766-e4ca5ca58591'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:18.196238 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-36453366-f052-4dbb-a766-e4ca5ca58591 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1eb2755befb44639cd693ab1a15c1ec', 'request_id': 'req-36453366-f052-4dbb-a766-e4ca5ca58591'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:18.211125 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-36453366-f052-4dbb-a766-e4ca5ca58591 admin admin] The token cache is being invalidate because project a1eb2755befb44639cd693ab1a15c1ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:32:18.211472 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-36453366-f052-4dbb-a766-e4ca5ca58591 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:18.212287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-36453366-f052-4dbb-a766-e4ca5ca58591 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:18.214544 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1371/2735] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:32:17 2022] DELETE /identity/v3/projects/a1eb2755befb44639cd693ab1a15c1ec => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:32:18.219495 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ef83c-0245-479e-9efb-cffdc2499533 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:18.219990 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ef83c-0245-479e-9efb-cffdc2499533 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:18.220084 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ef83c-0245-479e-9efb-cffdc2499533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:18.243027 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-4a9ef83c-0245-479e-9efb-cffdc2499533 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:32:18.244095 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-4a9ef83c-0245-479e-9efb-cffdc2499533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:18.289507 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:18.304640 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1365/2736] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:32:18.313229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526134f-7c18-449f-bb37-1fa280172cd7 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:18.313525 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526134f-7c18-449f-bb37-1fa280172cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:18.313841 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526134f-7c18-449f-bb37-1fa280172cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:18.337027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-2526134f-7c18-449f-bb37-1fa280172cd7 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:32:18.338621 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-2526134f-7c18-449f-bb37-1fa280172cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:18.350028 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1372/2737] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:32:18.352022 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ef83c-0245-479e-9efb-cffdc2499533 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:18.362223 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ef83c-0245-479e-9efb-cffdc2499533 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:18.367310 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8941cf8-b3a5-465a-845b-534eadd5498d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:18.367980 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8941cf8-b3a5-465a-845b-534eadd5498d admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:18.368266 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8941cf8-b3a5-465a-845b-534eadd5498d admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:18.368545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8941cf8-b3a5-465a-845b-534eadd5498d admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:18.369334 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8941cf8-b3a5-465a-845b-534eadd5498d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:32:18.369643 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8941cf8-b3a5-465a-845b-534eadd5498d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:18.371277 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8941cf8-b3a5-465a-845b-534eadd5498d admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:18.383775 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1366/2738] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:32:18.388180 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526134f-7c18-449f-bb37-1fa280172cd7 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:18.398834 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526134f-7c18-449f-bb37-1fa280172cd7 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:18.403779 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2911f-b73b-4d2e-ae02-735c05d0af0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:18.404457 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2911f-b73b-4d2e-ae02-735c05d0af0b admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:18.404730 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2911f-b73b-4d2e-ae02-735c05d0af0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:18.405038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2911f-b73b-4d2e-ae02-735c05d0af0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:18.405744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2911f-b73b-4d2e-ae02-735c05d0af0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:18.407769 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2911f-b73b-4d2e-ae02-735c05d0af0b admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:18.582865 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1373/2739] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:32:18 2022] POST /identity/v3/projects => generated 367 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:32:18.586875 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8941cf8-b3a5-465a-845b-534eadd5498d admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:18.597859 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8941cf8-b3a5-465a-845b-534eadd5498d admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:18.655949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afc99c-e94d-44c4-ae2b-851fa7bb67ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:18.656710 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afc99c-e94d-44c4-ae2b-851fa7bb67ee admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:18.657071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afc99c-e94d-44c4-ae2b-851fa7bb67ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:18.657423 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afc99c-e94d-44c4-ae2b-851fa7bb67ee admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:18.658323 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afc99c-e94d-44c4-ae2b-851fa7bb67ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:18.660384 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afc99c-e94d-44c4-ae2b-851fa7bb67ee admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:18.767002 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1367/2740] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:32:18 2022] POST /identity/v3/users => generated 284 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:32:18.771098 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2911f-b73b-4d2e-ae02-735c05d0af0b admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:18.783219 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2911f-b73b-4d2e-ae02-735c05d0af0b admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:18.788025 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd22a34-d3ea-44d3-a8c0-37477583d4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:18.788778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd22a34-d3ea-44d3-a8c0-37477583d4c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:18.789253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd22a34-d3ea-44d3-a8c0-37477583d4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:18.789543 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd22a34-d3ea-44d3-a8c0-37477583d4c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:18.790306 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd22a34-d3ea-44d3-a8c0-37477583d4c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:18.791986 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd22a34-d3ea-44d3-a8c0-37477583d4c7 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:18.802801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1374/2741] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:32:18.808808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afc99c-e94d-44c4-ae2b-851fa7bb67ee admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:18.819310 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afc99c-e94d-44c4-ae2b-851fa7bb67ee admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:18.824710 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e5ba5-a0d4-4296-ba95-204cf6535caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:18.825521 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e5ba5-a0d4-4296-ba95-204cf6535caa admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:18.829397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e5ba5-a0d4-4296-ba95-204cf6535caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:18.829397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e5ba5-a0d4-4296-ba95-204cf6535caa admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:18.829397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e5ba5-a0d4-4296-ba95-204cf6535caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:18.829397 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e5ba5-a0d4-4296-ba95-204cf6535caa admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:18.839202 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1368/2742] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:32:18.843219 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd22a34-d3ea-44d3-a8c0-37477583d4c7 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:18.853867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd22a34-d3ea-44d3-a8c0-37477583d4c7 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:18.858573 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f15ab-14fa-47e5-9e8b-10c92202a7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:18.859509 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f15ab-14fa-47e5-9e8b-10c92202a7d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:18.859801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f15ab-14fa-47e5-9e8b-10c92202a7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:18.862978 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f15ab-14fa-47e5-9e8b-10c92202a7d5 admin admin] PATH_INFO: `/v3/projects/285fec902a7c451c82e990b6ef678cdd/users/33a2717815614d6698a00856aec0b8ba/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:18.875794 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f15ab-14fa-47e5-9e8b-10c92202a7d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=285fec902a7c451c82e990b6ef678cdd, user_id=33a2717815614d6698a00856aec0b8ba, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:18.878661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f15ab-14fa-47e5-9e8b-10c92202a7d5 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:18.967021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1375/2743] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:32:18 2022] PUT /identity/v3/projects/285fec902a7c451c82e990b6ef678cdd/users/33a2717815614d6698a00856aec0b8ba/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:32:18.973069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e2320-250a-4725-8907-84854af1bb5e None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:18.973390 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e2320-250a-4725-8907-84854af1bb5e None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:18.973700 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e2320-250a-4725-8907-84854af1bb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:18.985301 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:19.003867 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-7b5e2320-250a-4725-8907-84854af1bb5e None None] MFA Rules not processed for user `33a2717815614d6698a00856aec0b8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:32:19.005099 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7b5e2320-250a-4725-8907-84854af1bb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:19.075459 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1369/2744] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:32:18 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:32:19.086788 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f15ab-14fa-47e5-9e8b-10c92202a7d5 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:19.088525 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d75f15ab-14fa-47e5-9e8b-10c92202a7d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:19.144440 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f15ab-14fa-47e5-9e8b-10c92202a7d5 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:19.148142 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684683c3-3793-4fc4-b342-f3e9a6933921 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36a6a3190dad4948a505bbe4c7095699', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:19.148856 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684683c3-3793-4fc4-b342-f3e9a6933921 service glance] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:19.149085 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684683c3-3793-4fc4-b342-f3e9a6933921 service glance] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:19.149366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684683c3-3793-4fc4-b342-f3e9a6933921 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:19.157928 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684683c3-3793-4fc4-b342-f3e9a6933921 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:19.159421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684683c3-3793-4fc4-b342-f3e9a6933921 service glance] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:19.169027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1376/2745] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:32:19 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:32:19.379451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:19.607596 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:19.791977 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e2320-250a-4725-8907-84854af1bb5e None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:19.793423 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7b5e2320-250a-4725-8907-84854af1bb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:19.863525 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e2320-250a-4725-8907-84854af1bb5e None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:19.869126 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b224631-a2a0-41ab-be35-c9dc3b5ac942 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63b20ceca40e6bceb56eda8ae9686', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:19.869781 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b224631-a2a0-41ab-be35-c9dc3b5ac942 service placement] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:19.870071 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b224631-a2a0-41ab-be35-c9dc3b5ac942 service placement] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:19.870354 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b224631-a2a0-41ab-be35-c9dc3b5ac942 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:19.872130 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-4b224631-a2a0-41ab-be35-c9dc3b5ac942 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:19.883623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b224631-a2a0-41ab-be35-c9dc3b5ac942 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:19.885245 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b224631-a2a0-41ab-be35-c9dc3b5ac942 service placement] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:19.895394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1370/2746] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:32:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:32:20.082739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:20.185030 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:20.750674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:20.939208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:21.112713 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:21.834870 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684683c3-3793-4fc4-b342-f3e9a6933921 service glance] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:21.836973 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-684683c3-3793-4fc4-b342-f3e9a6933921 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:21.899680 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684683c3-3793-4fc4-b342-f3e9a6933921 service glance] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:21.903631 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1186596-68a0-4054-990b-1adab93dbf4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:21.907131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1186596-68a0-4054-990b-1adab93dbf4f service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:21.907131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1186596-68a0-4054-990b-1adab93dbf4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:21.907131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1186596-68a0-4054-990b-1adab93dbf4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:21.907131 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d1186596-68a0-4054-990b-1adab93dbf4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:21.989080 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1186596-68a0-4054-990b-1adab93dbf4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:21.999396 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1186596-68a0-4054-990b-1adab93dbf4f service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:22.002476 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1377/2747] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:32:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:32:31.567136 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=8 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:31.997683 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=59 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:32:55.867395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b224631-a2a0-41ab-be35-c9dc3b5ac942 service placement] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:55.931970 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b224631-a2a0-41ab-be35-c9dc3b5ac942 service placement] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:55.936493 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda5cae-c82b-4598-a665-7c3c51bc8a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:55.937388 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda5cae-c82b-4598-a665-7c3c51bc8a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:55.937672 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda5cae-c82b-4598-a665-7c3c51bc8a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:55.937945 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda5cae-c82b-4598-a665-7c3c51bc8a6b admin admin] PATH_INFO: `/v3/users/33a2717815614d6698a00856aec0b8ba` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:55.939166 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda5cae-c82b-4598-a665-7c3c51bc8a6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=33a2717815614d6698a00856aec0b8ba)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:55.941077 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda5cae-c82b-4598-a665-7c3c51bc8a6b admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:56.222019 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-6cda5cae-c82b-4598-a665-7c3c51bc8a6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33a2717815614d6698a00856aec0b8ba'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:56.232627 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-6cda5cae-c82b-4598-a665-7c3c51bc8a6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33a2717815614d6698a00856aec0b8ba'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:56.233385 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-6cda5cae-c82b-4598-a665-7c3c51bc8a6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33a2717815614d6698a00856aec0b8ba'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:56.234003 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-6cda5cae-c82b-4598-a665-7c3c51bc8a6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33a2717815614d6698a00856aec0b8ba'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:56.251682 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-6cda5cae-c82b-4598-a665-7c3c51bc8a6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33a2717815614d6698a00856aec0b8ba'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:56.322152 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1371/2748] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:32:55 2022] DELETE /identity/v3/users/33a2717815614d6698a00856aec0b8ba => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:32:56.326764 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabde5d-6d57-460d-ab6d-8884ad583813 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:56.327321 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabde5d-6d57-460d-ab6d-8884ad583813 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:56.327525 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabde5d-6d57-460d-ab6d-8884ad583813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:56.345100 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-4aabde5d-6d57-460d-ab6d-8884ad583813 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:32:56.346308 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-4aabde5d-6d57-460d-ab6d-8884ad583813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:56.418578 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1378/2749] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:32:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:32:56.428681 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda5cae-c82b-4598-a665-7c3c51bc8a6b admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:56.430693 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-6cda5cae-c82b-4598-a665-7c3c51bc8a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:56.503723 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda5cae-c82b-4598-a665-7c3c51bc8a6b admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:56.507431 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65545a4e-3c29-4644-989e-bb6510e3e7eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:56.508087 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65545a4e-3c29-4644-989e-bb6510e3e7eb service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:56.508377 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65545a4e-3c29-4644-989e-bb6510e3e7eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:56.516849 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65545a4e-3c29-4644-989e-bb6510e3e7eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:56.524897 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65545a4e-3c29-4644-989e-bb6510e3e7eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:56.529325 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65545a4e-3c29-4644-989e-bb6510e3e7eb service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:56.537639 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1372/2750] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:32:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:32:57.003895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aabde5d-6d57-460d-ab6d-8884ad583813 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:32:57.005498 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-4aabde5d-6d57-460d-ab6d-8884ad583813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:32:57.026237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aabde5d-6d57-460d-ab6d-8884ad583813 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:32:57.026945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a026d0-ca6a-40dc-a9a8-6f7d63d4099e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:32:57.027974 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a026d0-ca6a-40dc-a9a8-6f7d63d4099e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:32:57.028330 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a026d0-ca6a-40dc-a9a8-6f7d63d4099e admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:32:57.028690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a026d0-ca6a-40dc-a9a8-6f7d63d4099e admin admin] PATH_INFO: `/v3/projects/285fec902a7c451c82e990b6ef678cdd` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:32:57.030255 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a026d0-ca6a-40dc-a9a8-6f7d63d4099e admin admin] RBAC: Authorizing `identity:delete_project(project_id=285fec902a7c451c82e990b6ef678cdd)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:32:57.032812 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a026d0-ca6a-40dc-a9a8-6f7d63d4099e admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:32:57.202998 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-35a026d0-ca6a-40dc-a9a8-6f7d63d4099e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '285fec902a7c451c82e990b6ef678cdd', 'request_id': 'req-35a026d0-ca6a-40dc-a9a8-6f7d63d4099e'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:57.209745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-35a026d0-ca6a-40dc-a9a8-6f7d63d4099e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '285fec902a7c451c82e990b6ef678cdd', 'request_id': 'req-35a026d0-ca6a-40dc-a9a8-6f7d63d4099e'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:57.300118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-35a026d0-ca6a-40dc-a9a8-6f7d63d4099e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '285fec902a7c451c82e990b6ef678cdd', 'request_id': 'req-35a026d0-ca6a-40dc-a9a8-6f7d63d4099e'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:57.315647 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-35a026d0-ca6a-40dc-a9a8-6f7d63d4099e admin admin] The token cache is being invalidate because project 285fec902a7c451c82e990b6ef678cdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:32:57.315647 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-35a026d0-ca6a-40dc-a9a8-6f7d63d4099e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:57.319723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-35a026d0-ca6a-40dc-a9a8-6f7d63d4099e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:32:57.319928 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1379/2751] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:32:57 2022] DELETE /identity/v3/projects/285fec902a7c451c82e990b6ef678cdd => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:33:00.749049 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d8bf1-fdd7-43fa-b933-227eac78a2c5 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:33:00.749049 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d8bf1-fdd7-43fa-b933-227eac78a2c5 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:33:00.749049 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d8bf1-fdd7-43fa-b933-227eac78a2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:33:00.765366 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-fb7d8bf1-fdd7-43fa-b933-227eac78a2c5 None None] MFA Rules not processed for user `9f0d4416e74a4126a6aa1ccf58e22ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:33:00.766769 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-fb7d8bf1-fdd7-43fa-b933-227eac78a2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:33:00.785360 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:33:00.822990 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1373/2752] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:33:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:33:07.309448 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:33:10.954745 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=15 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:33:37.075589 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559ce78-54be-45be-9229-ebeabf539404 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:33:37.076118 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559ce78-54be-45be-9229-ebeabf539404 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:33:37.076228 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559ce78-54be-45be-9229-ebeabf539404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:33:37.093618 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-d559ce78-54be-45be-9229-ebeabf539404 None None] MFA Rules not processed for user `ee3ad4a33625438fb5c0ae12314689d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:33:37.094849 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d559ce78-54be-45be-9229-ebeabf539404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:33:37.149624 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1380/2753] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:33:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:33:47.142966 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=25 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:33:58.969945 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d8bf1-fdd7-43fa-b933-227eac78a2c5 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:33:58.971375 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-fb7d8bf1-fdd7-43fa-b933-227eac78a2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:33:59.035566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d8bf1-fdd7-43fa-b933-227eac78a2c5 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:33:59.039678 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86890c51-13e3-4f16-af79-e7cc32d9360f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:33:59.040329 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86890c51-13e3-4f16-af79-e7cc32d9360f service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:33:59.040624 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86890c51-13e3-4f16-af79-e7cc32d9360f service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:33:59.040890 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86890c51-13e3-4f16-af79-e7cc32d9360f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:33:59.042674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-86890c51-13e3-4f16-af79-e7cc32d9360f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:33:59.114399 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86890c51-13e3-4f16-af79-e7cc32d9360f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:33:59.116137 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86890c51-13e3-4f16-af79-e7cc32d9360f service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:33:59.127767 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1374/2754] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:33:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:34:09.122818 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=54 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:11.334116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559ce78-54be-45be-9229-ebeabf539404 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:11.336154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-d559ce78-54be-45be-9229-ebeabf539404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:11.348185 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559ce78-54be-45be-9229-ebeabf539404 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:11.352523 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31d728-fd4f-4508-ba43-6893774d14ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:11.353220 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31d728-fd4f-4508-ba43-6893774d14ee service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:11.353508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31d728-fd4f-4508-ba43-6893774d14ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:11.353785 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31d728-fd4f-4508-ba43-6893774d14ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:11.355746 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-3d31d728-fd4f-4508-ba43-6893774d14ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:11.369867 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31d728-fd4f-4508-ba43-6893774d14ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:11.371345 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31d728-fd4f-4508-ba43-6893774d14ee service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:11.382723 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1381/2755] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:34:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:34:21.377587 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=6 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:30.904949 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86890c51-13e3-4f16-af79-e7cc32d9360f service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:30.906366 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-86890c51-13e3-4f16-af79-e7cc32d9360f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:30.920322 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86890c51-13e3-4f16-af79-e7cc32d9360f service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:30.925808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5450b-6c65-4f50-abbf-4716ce8d41ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:30.926461 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5450b-6c65-4f50-abbf-4716ce8d41ee service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:30.926753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5450b-6c65-4f50-abbf-4716ce8d41ee service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:30.927078 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5450b-6c65-4f50-abbf-4716ce8d41ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:30.930853 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-abe5450b-6c65-4f50-abbf-4716ce8d41ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:30.943378 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5450b-6c65-4f50-abbf-4716ce8d41ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:30.944990 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5450b-6c65-4f50-abbf-4716ce8d41ee service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:30.954732 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1375/2756] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:34:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:34:37.658652 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31d728-fd4f-4508-ba43-6893774d14ee service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:37.662238 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-3d31d728-fd4f-4508-ba43-6893774d14ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:37.679295 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31d728-fd4f-4508-ba43-6893774d14ee service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:37.685852 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8bf2c5-5b7f-4a91-8484-68f93ba68136 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:37.686775 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bf2c5-5b7f-4a91-8484-68f93ba68136 service nova] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:37.687275 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bf2c5-5b7f-4a91-8484-68f93ba68136 service nova] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:37.687834 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bf2c5-5b7f-4a91-8484-68f93ba68136 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:37.690057 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-cc8bf2c5-5b7f-4a91-8484-68f93ba68136 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:37.712834 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8bf2c5-5b7f-4a91-8484-68f93ba68136 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:37.715386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8bf2c5-5b7f-4a91-8484-68f93ba68136 service nova] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:37.734834 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1382/2757] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:34:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:34:39.730803 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acaebf-10a7-4837-85a9-6b4aa1d8fd39 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:39.731227 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acaebf-10a7-4837-85a9-6b4aa1d8fd39 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:39.731339 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acaebf-10a7-4837-85a9-6b4aa1d8fd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:39.750567 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-d1acaebf-10a7-4837-85a9-6b4aa1d8fd39 None None] MFA Rules not processed for user `86a640cdceb34bb9bfbb7b769e797d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:34:39.751824 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-d1acaebf-10a7-4837-85a9-6b4aa1d8fd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:39.807574 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1376/2758] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:34:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:34:40.119348 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8bf2c5-5b7f-4a91-8484-68f93ba68136 service nova] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:40.120841 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-cc8bf2c5-5b7f-4a91-8484-68f93ba68136 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:40.183544 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8bf2c5-5b7f-4a91-8484-68f93ba68136 service nova] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:40.187390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f66699-1275-4b2d-8c94-02ecb5f23f12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:40.188044 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f66699-1275-4b2d-8c94-02ecb5f23f12 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:40.188333 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f66699-1275-4b2d-8c94-02ecb5f23f12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:40.189038 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f66699-1275-4b2d-8c94-02ecb5f23f12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:40.190471 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-b7f66699-1275-4b2d-8c94-02ecb5f23f12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:40.201149 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f66699-1275-4b2d-8c94-02ecb5f23f12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:40.202587 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f66699-1275-4b2d-8c94-02ecb5f23f12 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:40.213178 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1383/2759] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:34:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:34:41.023843 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1acaebf-10a7-4837-85a9-6b4aa1d8fd39 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:41.025337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-d1acaebf-10a7-4837-85a9-6b4aa1d8fd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:41.100545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1acaebf-10a7-4837-85a9-6b4aa1d8fd39 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:41.106164 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc9320-d352-413f-ba96-74ca08d59d46 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63b20ceca40e6bceb56eda8ae9686', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:41.106823 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc9320-d352-413f-ba96-74ca08d59d46 service placement] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:41.107144 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc9320-d352-413f-ba96-74ca08d59d46 service placement] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:41.110444 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc9320-d352-413f-ba96-74ca08d59d46 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:41.110444 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-d1dc9320-d352-413f-ba96-74ca08d59d46 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:41.121916 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc9320-d352-413f-ba96-74ca08d59d46 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:41.123693 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc9320-d352-413f-ba96-74ca08d59d46 service placement] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:41.133825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1377/2760] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:34:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:34:42.003338 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f66699-1275-4b2d-8c94-02ecb5f23f12 service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:42.012894 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f66699-1275-4b2d-8c94-02ecb5f23f12 service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:42.016707 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48f6ea-55fa-4d63-8f2b-859c5b88a850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:42.017337 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48f6ea-55fa-4d63-8f2b-859c5b88a850 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:42.017622 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48f6ea-55fa-4d63-8f2b-859c5b88a850 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:42.017940 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48f6ea-55fa-4d63-8f2b-859c5b88a850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:42.019783 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-4d48f6ea-55fa-4d63-8f2b-859c5b88a850 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:42.087935 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d48f6ea-55fa-4d63-8f2b-859c5b88a850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:42.089507 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d48f6ea-55fa-4d63-8f2b-859c5b88a850 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:42.099614 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1384/2761] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:34:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:34:44.156772 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc9320-d352-413f-ba96-74ca08d59d46 service placement] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:44.158491 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-d1dc9320-d352-413f-ba96-74ca08d59d46 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:44.174761 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc9320-d352-413f-ba96-74ca08d59d46 service placement] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:44.180207 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc007cbf-12a5-4628-91fd-9e6c3b81bc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:44.181162 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc007cbf-12a5-4628-91fd-9e6c3b81bc33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:44.181585 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc007cbf-12a5-4628-91fd-9e6c3b81bc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:44.182305 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc007cbf-12a5-4628-91fd-9e6c3b81bc33 admin admin] PATH_INFO: `/v3/users/ee3ad4a33625438fb5c0ae12314689d6` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:44.183765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc007cbf-12a5-4628-91fd-9e6c3b81bc33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee3ad4a33625438fb5c0ae12314689d6)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:44.185881 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc007cbf-12a5-4628-91fd-9e6c3b81bc33 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:44.465563 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-dc007cbf-12a5-4628-91fd-9e6c3b81bc33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee3ad4a33625438fb5c0ae12314689d6'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:44.481982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-dc007cbf-12a5-4628-91fd-9e6c3b81bc33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee3ad4a33625438fb5c0ae12314689d6'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:44.483741 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-dc007cbf-12a5-4628-91fd-9e6c3b81bc33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee3ad4a33625438fb5c0ae12314689d6'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:44.483741 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-dc007cbf-12a5-4628-91fd-9e6c3b81bc33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee3ad4a33625438fb5c0ae12314689d6'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:44.508465 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-dc007cbf-12a5-4628-91fd-9e6c3b81bc33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee3ad4a33625438fb5c0ae12314689d6'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:44.516384 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:44.589806 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1378/2762] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:34:44 2022] DELETE /identity/v3/users/ee3ad4a33625438fb5c0ae12314689d6 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:34:44.937076 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48f6ea-55fa-4d63-8f2b-859c5b88a850 service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:44.951823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48f6ea-55fa-4d63-8f2b-859c5b88a850 service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:45.041855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8dc65e-d770-4178-8223-3527656ac136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:45.043113 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8dc65e-d770-4178-8223-3527656ac136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:45.043850 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8dc65e-d770-4178-8223-3527656ac136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:45.044297 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8dc65e-d770-4178-8223-3527656ac136 admin admin] PATH_INFO: `/v3/projects/e06cabc6fd104c7cbd2758700a0d29a8` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:45.047046 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8dc65e-d770-4178-8223-3527656ac136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e06cabc6fd104c7cbd2758700a0d29a8)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:45.049277 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8dc65e-d770-4178-8223-3527656ac136 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:45.201534 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc007cbf-12a5-4628-91fd-9e6c3b81bc33 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:45.203134 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:45.204209 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-dc007cbf-12a5-4628-91fd-9e6c3b81bc33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:45.224451 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4f8dc65e-d770-4178-8223-3527656ac136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e06cabc6fd104c7cbd2758700a0d29a8', 'request_id': 'req-4f8dc65e-d770-4178-8223-3527656ac136'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:45.233032 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4f8dc65e-d770-4178-8223-3527656ac136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e06cabc6fd104c7cbd2758700a0d29a8', 'request_id': 'req-4f8dc65e-d770-4178-8223-3527656ac136'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:45.277125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc007cbf-12a5-4628-91fd-9e6c3b81bc33 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:45.327355 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a85d5-fb23-4948-bce6-9d41173d5076 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:45.328026 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a85d5-fb23-4948-bce6-9d41173d5076 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:45.328319 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a85d5-fb23-4948-bce6-9d41173d5076 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:45.328604 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a85d5-fb23-4948-bce6-9d41173d5076 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:45.330406 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-dd2a85d5-fb23-4948-bce6-9d41173d5076 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:45.332525 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4f8dc65e-d770-4178-8223-3527656ac136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e06cabc6fd104c7cbd2758700a0d29a8', 'request_id': 'req-4f8dc65e-d770-4178-8223-3527656ac136'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:45.342172 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a85d5-fb23-4948-bce6-9d41173d5076 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:45.343720 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a85d5-fb23-4948-bce6-9d41173d5076 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:45.344643 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4f8dc65e-d770-4178-8223-3527656ac136 admin admin] The token cache is being invalidate because project e06cabc6fd104c7cbd2758700a0d29a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:34:45.344975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4f8dc65e-d770-4178-8223-3527656ac136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:45.345677 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4f8dc65e-d770-4178-8223-3527656ac136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:45.347552 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1385/2763] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:34:44 2022] DELETE /identity/v3/projects/e06cabc6fd104c7cbd2758700a0d29a8 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:34:45.351977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79701727-526a-450f-a4da-236bcdc067b6 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:45.352347 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79701727-526a-450f-a4da-236bcdc067b6 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:45.352716 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79701727-526a-450f-a4da-236bcdc067b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:45.353607 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1379/2764] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:34:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:34:45.370887 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-79701727-526a-450f-a4da-236bcdc067b6 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:34:45.372120 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-79701727-526a-450f-a4da-236bcdc067b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:45.431775 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1386/2765] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:34:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:34:45.439887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f43251-733a-405d-8698-47368497d2cd None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:45.439887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f43251-733a-405d-8698-47368497d2cd None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:45.439887 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f43251-733a-405d-8698-47368497d2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:45.461517 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-e7f43251-733a-405d-8698-47368497d2cd None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:34:45.463085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-e7f43251-733a-405d-8698-47368497d2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:45.478085 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79701727-526a-450f-a4da-236bcdc067b6 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:45.485021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1380/2766] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:34:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:34:45.489166 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79701727-526a-450f-a4da-236bcdc067b6 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:45.496051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b15880-21fd-4826-9699-a65b34324965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:45.497026 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b15880-21fd-4826-9699-a65b34324965 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:45.497505 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b15880-21fd-4826-9699-a65b34324965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:45.497993 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b15880-21fd-4826-9699-a65b34324965 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:45.498975 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b15880-21fd-4826-9699-a65b34324965 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:34:45.499476 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b15880-21fd-4826-9699-a65b34324965 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:45.501568 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b15880-21fd-4826-9699-a65b34324965 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:45.529653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1387/2767] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:34:45.534632 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f43251-733a-405d-8698-47368497d2cd None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:45.546109 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f43251-733a-405d-8698-47368497d2cd None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:45.551509 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722147fa-f86e-48a2-9eac-46889f2ce43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:45.552707 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722147fa-f86e-48a2-9eac-46889f2ce43d admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:45.553239 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722147fa-f86e-48a2-9eac-46889f2ce43d admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:45.554079 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722147fa-f86e-48a2-9eac-46889f2ce43d admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:45.556178 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722147fa-f86e-48a2-9eac-46889f2ce43d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:45.559846 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722147fa-f86e-48a2-9eac-46889f2ce43d admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:45.667378 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1381/2768] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:34:45 2022] POST /identity/v3/projects => generated 397 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:34:45.671307 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b15880-21fd-4826-9699-a65b34324965 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:45.685505 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b15880-21fd-4826-9699-a65b34324965 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:45.759752 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8ced6-9f69-44da-8bec-9de225fab74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:45.760536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8ced6-9f69-44da-8bec-9de225fab74b admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:45.760893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8ced6-9f69-44da-8bec-9de225fab74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:45.761239 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8ced6-9f69-44da-8bec-9de225fab74b admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:45.762206 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8ced6-9f69-44da-8bec-9de225fab74b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:45.764823 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8ced6-9f69-44da-8bec-9de225fab74b admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:45.857844 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1388/2769] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:34:45 2022] POST /identity/v3/users => generated 299 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:34:45.861541 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722147fa-f86e-48a2-9eac-46889f2ce43d admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:45.879569 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722147fa-f86e-48a2-9eac-46889f2ce43d admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:45.888835 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582758fd-6322-4c66-95af-8231f635239f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:45.889057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582758fd-6322-4c66-95af-8231f635239f admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:45.889698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582758fd-6322-4c66-95af-8231f635239f admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:45.889698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582758fd-6322-4c66-95af-8231f635239f admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:45.890270 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582758fd-6322-4c66-95af-8231f635239f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:45.891962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582758fd-6322-4c66-95af-8231f635239f admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:45.910512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1382/2770] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:34:45.915130 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8ced6-9f69-44da-8bec-9de225fab74b admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:45.926724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8ced6-9f69-44da-8bec-9de225fab74b admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:45.940607 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf852a15-cba8-4b78-9aa0-cccc55de49a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:45.941500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf852a15-cba8-4b78-9aa0-cccc55de49a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:45.942943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf852a15-cba8-4b78-9aa0-cccc55de49a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:45.942943 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf852a15-cba8-4b78-9aa0-cccc55de49a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:45.944045 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf852a15-cba8-4b78-9aa0-cccc55de49a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:45.946701 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf852a15-cba8-4b78-9aa0-cccc55de49a5 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:45.975057 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1389/2771] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:34:45.982105 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582758fd-6322-4c66-95af-8231f635239f admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:46.009165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582758fd-6322-4c66-95af-8231f635239f admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:46.020383 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d2076-0d92-49b7-898d-ce0d6b0d3e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:46.021406 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d2076-0d92-49b7-898d-ce0d6b0d3e41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:46.021769 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d2076-0d92-49b7-898d-ce0d6b0d3e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:46.022147 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d2076-0d92-49b7-898d-ce0d6b0d3e41 admin admin] PATH_INFO: `/v3/projects/05bdf101dc8c4d22b770d3565fc5d2ce/users/b1ec74f16cf147c0a799864bbe280ac8/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:46.044793 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d2076-0d92-49b7-898d-ce0d6b0d3e41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05bdf101dc8c4d22b770d3565fc5d2ce, user_id=b1ec74f16cf147c0a799864bbe280ac8, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:46.049165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d2076-0d92-49b7-898d-ce0d6b0d3e41 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:46.160414 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1383/2772] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:34:45 2022] PUT /identity/v3/projects/05bdf101dc8c4d22b770d3565fc5d2ce/users/b1ec74f16cf147c0a799864bbe280ac8/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:34:46.173820 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cde92-8d6a-4a87-8189-32f819a8ff87 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:46.174114 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cde92-8d6a-4a87-8189-32f819a8ff87 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:46.174443 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cde92-8d6a-4a87-8189-32f819a8ff87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:46.200933 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-610cde92-8d6a-4a87-8189-32f819a8ff87 None None] MFA Rules not processed for user `b1ec74f16cf147c0a799864bbe280ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:34:46.202126 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-610cde92-8d6a-4a87-8189-32f819a8ff87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:46.210250 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:46.276083 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1390/2773] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:34:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:34:46.288454 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d2076-0d92-49b7-898d-ce0d6b0d3e41 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:46.290787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-ff2d2076-0d92-49b7-898d-ce0d6b0d3e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:46.375733 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:46.386539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d2076-0d92-49b7-898d-ce0d6b0d3e41 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:46.395743 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ff678-5e76-47a0-bdca-766d5bc1ab23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:46.396505 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ff678-5e76-47a0-bdca-766d5bc1ab23 service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:46.396876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ff678-5e76-47a0-bdca-766d5bc1ab23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:46.397228 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ff678-5e76-47a0-bdca-766d5bc1ab23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:46.408658 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ff678-5e76-47a0-bdca-766d5bc1ab23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:46.410231 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ff678-5e76-47a0-bdca-766d5bc1ab23 service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:46.431769 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1384/2774] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:34:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:34:47.286588 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610cde92-8d6a-4a87-8189-32f819a8ff87 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:47.289312 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-610cde92-8d6a-4a87-8189-32f819a8ff87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:47.358591 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610cde92-8d6a-4a87-8189-32f819a8ff87 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:47.363546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9bc34b-3d8f-48aa-a970-e54e110fe51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:47.364756 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bc34b-3d8f-48aa-a970-e54e110fe51e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:47.365233 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bc34b-3d8f-48aa-a970-e54e110fe51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:47.365513 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bc34b-3d8f-48aa-a970-e54e110fe51e admin admin] PATH_INFO: `/v3/users/9f0d4416e74a4126a6aa1ccf58e22ee3` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:47.367570 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9bc34b-3d8f-48aa-a970-e54e110fe51e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f0d4416e74a4126a6aa1ccf58e22ee3)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:47.369498 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9bc34b-3d8f-48aa-a970-e54e110fe51e admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:47.672778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-ec9bc34b-3d8f-48aa-a970-e54e110fe51e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f0d4416e74a4126a6aa1ccf58e22ee3'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:47.680935 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-ec9bc34b-3d8f-48aa-a970-e54e110fe51e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f0d4416e74a4126a6aa1ccf58e22ee3'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:47.681873 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-ec9bc34b-3d8f-48aa-a970-e54e110fe51e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f0d4416e74a4126a6aa1ccf58e22ee3'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:47.682560 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-ec9bc34b-3d8f-48aa-a970-e54e110fe51e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f0d4416e74a4126a6aa1ccf58e22ee3'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:47.696235 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-ec9bc34b-3d8f-48aa-a970-e54e110fe51e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f0d4416e74a4126a6aa1ccf58e22ee3'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:47.864518 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1391/2775] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:34:47 2022] DELETE /identity/v3/users/9f0d4416e74a4126a6aa1ccf58e22ee3 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:34:48.110825 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ff678-5e76-47a0-bdca-766d5bc1ab23 service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:48.126294 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ff678-5e76-47a0-bdca-766d5bc1ab23 service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:48.192199 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887cbb4-b51d-442c-87dc-e6d0ac690e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:48.193070 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887cbb4-b51d-442c-87dc-e6d0ac690e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:48.193394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887cbb4-b51d-442c-87dc-e6d0ac690e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:48.194008 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887cbb4-b51d-442c-87dc-e6d0ac690e44 admin admin] PATH_INFO: `/v3/projects/19a424690e5b44d58c27c5f872c38bd4` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:48.195816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887cbb4-b51d-442c-87dc-e6d0ac690e44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19a424690e5b44d58c27c5f872c38bd4)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:48.197812 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887cbb4-b51d-442c-87dc-e6d0ac690e44 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:48.320904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-4887cbb4-b51d-442c-87dc-e6d0ac690e44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19a424690e5b44d58c27c5f872c38bd4', 'request_id': 'req-4887cbb4-b51d-442c-87dc-e6d0ac690e44'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:48.327340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-4887cbb4-b51d-442c-87dc-e6d0ac690e44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19a424690e5b44d58c27c5f872c38bd4', 'request_id': 'req-4887cbb4-b51d-442c-87dc-e6d0ac690e44'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:48.332981 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-4887cbb4-b51d-442c-87dc-e6d0ac690e44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19a424690e5b44d58c27c5f872c38bd4', 'request_id': 'req-4887cbb4-b51d-442c-87dc-e6d0ac690e44'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:48.400257 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-4887cbb4-b51d-442c-87dc-e6d0ac690e44 admin admin] The token cache is being invalidate because project 19a424690e5b44d58c27c5f872c38bd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:34:48.400531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-4887cbb4-b51d-442c-87dc-e6d0ac690e44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:48.401096 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-4887cbb4-b51d-442c-87dc-e6d0ac690e44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:48.402696 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1385/2776] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:34:48 2022] DELETE /identity/v3/projects/19a424690e5b44d58c27c5f872c38bd4 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:34:48.407683 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b22613-cd3a-4c8a-8ec3-7abd5e57fbd7 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:48.408192 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b22613-cd3a-4c8a-8ec3-7abd5e57fbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:48.408318 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b22613-cd3a-4c8a-8ec3-7abd5e57fbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:48.426410 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-47b22613-cd3a-4c8a-8ec3-7abd5e57fbd7 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:34:48.427693 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-47b22613-cd3a-4c8a-8ec3-7abd5e57fbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:48.454683 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:48.490025 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1392/2777] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:34:48.498818 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add39edd-0fc0-4506-a339-2b6f185bd09c None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:48.499147 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add39edd-0fc0-4506-a339-2b6f185bd09c None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:48.499434 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add39edd-0fc0-4506-a339-2b6f185bd09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:48.519240 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-add39edd-0fc0-4506-a339-2b6f185bd09c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:34:48.520446 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-add39edd-0fc0-4506-a339-2b6f185bd09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:48.528607 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1386/2778] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:34:48.532965 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b22613-cd3a-4c8a-8ec3-7abd5e57fbd7 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:48.544324 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b22613-cd3a-4c8a-8ec3-7abd5e57fbd7 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:48.549016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860caa8-042f-4f8b-bd4a-8129f75eaf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:48.549681 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860caa8-042f-4f8b-bd4a-8129f75eaf22 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:48.549970 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860caa8-042f-4f8b-bd4a-8129f75eaf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:48.550251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860caa8-042f-4f8b-bd4a-8129f75eaf22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:48.551066 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860caa8-042f-4f8b-bd4a-8129f75eaf22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:34:48.551382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860caa8-042f-4f8b-bd4a-8129f75eaf22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:48.553154 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860caa8-042f-4f8b-bd4a-8129f75eaf22 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:48.565783 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1393/2779] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:34:48.570117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add39edd-0fc0-4506-a339-2b6f185bd09c None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:48.579407 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add39edd-0fc0-4506-a339-2b6f185bd09c None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:48.584305 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86756c-1c5f-4751-bea3-504865e8e135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:48.584986 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86756c-1c5f-4751-bea3-504865e8e135 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:48.585275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86756c-1c5f-4751-bea3-504865e8e135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:48.585555 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86756c-1c5f-4751-bea3-504865e8e135 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:48.586314 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e86756c-1c5f-4751-bea3-504865e8e135 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:48.588253 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e86756c-1c5f-4751-bea3-504865e8e135 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:48.658203 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1387/2780] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:34:48 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:34:48.662021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860caa8-042f-4f8b-bd4a-8129f75eaf22 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:48.672576 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860caa8-042f-4f8b-bd4a-8129f75eaf22 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:48.729013 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012b0be-7269-46c8-a926-8da849a7fb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:48.729750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012b0be-7269-46c8-a926-8da849a7fb4b admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:48.730032 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012b0be-7269-46c8-a926-8da849a7fb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:48.730310 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012b0be-7269-46c8-a926-8da849a7fb4b admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:48.731069 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012b0be-7269-46c8-a926-8da849a7fb4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:48.732993 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012b0be-7269-46c8-a926-8da849a7fb4b admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:48.864012 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1394/2781] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:34:48 2022] POST /identity/v3/users => generated 287 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:34:48.868359 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86756c-1c5f-4751-bea3-504865e8e135 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:48.878492 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86756c-1c5f-4751-bea3-504865e8e135 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:48.883070 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7ca27-1573-40db-8bb6-a281e6b2b08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:48.883794 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7ca27-1573-40db-8bb6-a281e6b2b08d admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:48.884085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7ca27-1573-40db-8bb6-a281e6b2b08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:48.884361 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7ca27-1573-40db-8bb6-a281e6b2b08d admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:48.885131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7ca27-1573-40db-8bb6-a281e6b2b08d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:48.886719 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7ca27-1573-40db-8bb6-a281e6b2b08d admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:48.897053 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1388/2782] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:34:48.901389 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012b0be-7269-46c8-a926-8da849a7fb4b admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:48.911382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012b0be-7269-46c8-a926-8da849a7fb4b admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:48.916746 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8bb24-8038-4736-902a-e89dc464ceaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:48.917381 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8bb24-8038-4736-902a-e89dc464ceaf admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:48.917674 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8bb24-8038-4736-902a-e89dc464ceaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:48.917959 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8bb24-8038-4736-902a-e89dc464ceaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:48.918693 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e8bb24-8038-4736-902a-e89dc464ceaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:48.920386 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e8bb24-8038-4736-902a-e89dc464ceaf admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:48.930561 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1395/2783] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:34:48.934790 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7ca27-1573-40db-8bb6-a281e6b2b08d admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:48.944659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7ca27-1573-40db-8bb6-a281e6b2b08d admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:48.954565 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be57ca6-b9ab-41c9-a591-6077d63f985d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:48.955514 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be57ca6-b9ab-41c9-a591-6077d63f985d admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:48.957492 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be57ca6-b9ab-41c9-a591-6077d63f985d admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:48.957492 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be57ca6-b9ab-41c9-a591-6077d63f985d admin admin] PATH_INFO: `/v3/projects/6b5047101a0142e1ab06e6a2ec39cf2f/users/73c41cfe5eb04ea1b9c2c175ea84215c/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:48.974885 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be57ca6-b9ab-41c9-a591-6077d63f985d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b5047101a0142e1ab06e6a2ec39cf2f, user_id=73c41cfe5eb04ea1b9c2c175ea84215c, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:48.978249 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be57ca6-b9ab-41c9-a591-6077d63f985d admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:49.130794 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1389/2784] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:34:48 2022] PUT /identity/v3/projects/6b5047101a0142e1ab06e6a2ec39cf2f/users/73c41cfe5eb04ea1b9c2c175ea84215c/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:34:49.141582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257176cc-e4a9-478b-a96f-75558b890692 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:49.141879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257176cc-e4a9-478b-a96f-75558b890692 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:49.142157 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257176cc-e4a9-478b-a96f-75558b890692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:49.170074 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-257176cc-e4a9-478b-a96f-75558b890692 None None] MFA Rules not processed for user `73c41cfe5eb04ea1b9c2c175ea84215c`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:34:49.171261 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-257176cc-e4a9-478b-a96f-75558b890692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:49.235488 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1396/2785] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:34:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:34:49.246687 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be57ca6-b9ab-41c9-a591-6077d63f985d admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:49.248444 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7be57ca6-b9ab-41c9-a591-6077d63f985d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:49.269030 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:49.323723 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be57ca6-b9ab-41c9-a591-6077d63f985d admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:49.330337 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6f332-8172-402a-b236-c2ddf9bd5eef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:49.331021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6f332-8172-402a-b236-c2ddf9bd5eef service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:49.331310 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6f332-8172-402a-b236-c2ddf9bd5eef service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:49.331586 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6f332-8172-402a-b236-c2ddf9bd5eef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:49.340367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6f332-8172-402a-b236-c2ddf9bd5eef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:49.341947 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6f332-8172-402a-b236-c2ddf9bd5eef service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:49.352404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1390/2786] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:34:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:34:49.575329 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257176cc-e4a9-478b-a96f-75558b890692 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:49.601793 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:49.637562 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257176cc-e4a9-478b-a96f-75558b890692 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:49.642173 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3edf9-0edb-4246-ba60-5cf515d06042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:49.643103 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3edf9-0edb-4246-ba60-5cf515d06042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:49.643392 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3edf9-0edb-4246-ba60-5cf515d06042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:49.643668 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3edf9-0edb-4246-ba60-5cf515d06042 admin admin] PATH_INFO: `/v3/users/73c41cfe5eb04ea1b9c2c175ea84215c` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:49.644910 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3edf9-0edb-4246-ba60-5cf515d06042 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73c41cfe5eb04ea1b9c2c175ea84215c)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:49.646841 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3edf9-0edb-4246-ba60-5cf515d06042 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:49.854963 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-37d3edf9-0edb-4246-ba60-5cf515d06042 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73c41cfe5eb04ea1b9c2c175ea84215c'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:49.861743 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-37d3edf9-0edb-4246-ba60-5cf515d06042 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73c41cfe5eb04ea1b9c2c175ea84215c'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:49.862411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-37d3edf9-0edb-4246-ba60-5cf515d06042 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73c41cfe5eb04ea1b9c2c175ea84215c'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:49.863075 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-37d3edf9-0edb-4246-ba60-5cf515d06042 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73c41cfe5eb04ea1b9c2c175ea84215c'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:49.875546 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-37d3edf9-0edb-4246-ba60-5cf515d06042 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73c41cfe5eb04ea1b9c2c175ea84215c'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:49.943340 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1397/2787] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:34:49 2022] DELETE /identity/v3/users/73c41cfe5eb04ea1b9c2c175ea84215c => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:34:49.948030 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60874ff7-1074-42cb-88ae-5cd0551afafd None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:49.948181 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60874ff7-1074-42cb-88ae-5cd0551afafd None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:49.949213 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60874ff7-1074-42cb-88ae-5cd0551afafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:49.965692 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-60874ff7-1074-42cb-88ae-5cd0551afafd None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:34:49.966941 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-60874ff7-1074-42cb-88ae-5cd0551afafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:50.028687 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1391/2788] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:34:50.038097 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3edf9-0edb-4246-ba60-5cf515d06042 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:50.040050 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-37d3edf9-0edb-4246-ba60-5cf515d06042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:50.095662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3edf9-0edb-4246-ba60-5cf515d06042 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:50.099244 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9766c05-5bde-473f-81e4-17a385ae760c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:50.099879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9766c05-5bde-473f-81e4-17a385ae760c service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:50.100165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9766c05-5bde-473f-81e4-17a385ae760c service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:50.100438 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9766c05-5bde-473f-81e4-17a385ae760c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:50.109782 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9766c05-5bde-473f-81e4-17a385ae760c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:50.111247 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9766c05-5bde-473f-81e4-17a385ae760c service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:50.121474 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1398/2789] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:34:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:34:50.520500 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60874ff7-1074-42cb-88ae-5cd0551afafd None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:50.521992 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-60874ff7-1074-42cb-88ae-5cd0551afafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:50.537772 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60874ff7-1074-42cb-88ae-5cd0551afafd None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:50.544200 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f12ac-1f26-4e3b-a39d-48463fc4e4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:50.545402 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f12ac-1f26-4e3b-a39d-48463fc4e4d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:50.545677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f12ac-1f26-4e3b-a39d-48463fc4e4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:50.546283 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f12ac-1f26-4e3b-a39d-48463fc4e4d0 admin admin] PATH_INFO: `/v3/projects/6b5047101a0142e1ab06e6a2ec39cf2f` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:50.547365 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f12ac-1f26-4e3b-a39d-48463fc4e4d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b5047101a0142e1ab06e6a2ec39cf2f)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:50.549670 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f12ac-1f26-4e3b-a39d-48463fc4e4d0 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:50.646793 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-2e5f12ac-1f26-4e3b-a39d-48463fc4e4d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b5047101a0142e1ab06e6a2ec39cf2f', 'request_id': 'req-2e5f12ac-1f26-4e3b-a39d-48463fc4e4d0'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:50.653021 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-2e5f12ac-1f26-4e3b-a39d-48463fc4e4d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b5047101a0142e1ab06e6a2ec39cf2f', 'request_id': 'req-2e5f12ac-1f26-4e3b-a39d-48463fc4e4d0'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:50.658551 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-2e5f12ac-1f26-4e3b-a39d-48463fc4e4d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b5047101a0142e1ab06e6a2ec39cf2f', 'request_id': 'req-2e5f12ac-1f26-4e3b-a39d-48463fc4e4d0'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:50.748012 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-2e5f12ac-1f26-4e3b-a39d-48463fc4e4d0 admin admin] The token cache is being invalidate because project 6b5047101a0142e1ab06e6a2ec39cf2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:34:50.748308 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-2e5f12ac-1f26-4e3b-a39d-48463fc4e4d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:50.748948 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-2e5f12ac-1f26-4e3b-a39d-48463fc4e4d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:50.750965 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1392/2790] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:34:50 2022] DELETE /identity/v3/projects/6b5047101a0142e1ab06e6a2ec39cf2f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:34:50.756449 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8def3-e9c4-4a51-ba9b-23566f0ae9fe None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:50.763748 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8def3-e9c4-4a51-ba9b-23566f0ae9fe None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:50.763748 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8def3-e9c4-4a51-ba9b-23566f0ae9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:50.774772 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-16f8def3-e9c4-4a51-ba9b-23566f0ae9fe None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:34:50.776051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-16f8def3-e9c4-4a51-ba9b-23566f0ae9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:50.818427 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:50.835239 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1399/2791] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:34:50.843182 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c6f06-a41a-4821-9a15-d9b3010e377f None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:50.847983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c6f06-a41a-4821-9a15-d9b3010e377f None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:50.847983 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c6f06-a41a-4821-9a15-d9b3010e377f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:50.862128 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-a06c6f06-a41a-4821-9a15-d9b3010e377f None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:34:50.863356 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-a06c6f06-a41a-4821-9a15-d9b3010e377f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:50.871976 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1393/2792] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:34:50.875962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8def3-e9c4-4a51-ba9b-23566f0ae9fe None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:50.885288 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8def3-e9c4-4a51-ba9b-23566f0ae9fe None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:50.889864 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0ddc3-5521-4500-9707-0c012113c623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:50.890516 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0ddc3-5521-4500-9707-0c012113c623 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:50.893870 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0ddc3-5521-4500-9707-0c012113c623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:50.893870 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0ddc3-5521-4500-9707-0c012113c623 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:50.893870 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0ddc3-5521-4500-9707-0c012113c623 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:34:50.893870 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0ddc3-5521-4500-9707-0c012113c623 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:50.893870 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0ddc3-5521-4500-9707-0c012113c623 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:50.906000 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1400/2793] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:34:50.909996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c6f06-a41a-4821-9a15-d9b3010e377f None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:50.919847 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c6f06-a41a-4821-9a15-d9b3010e377f None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:50.924471 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1a9de-7f92-4b23-9fb6-24efdc822394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:50.925118 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1a9de-7f92-4b23-9fb6-24efdc822394 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:50.925404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1a9de-7f92-4b23-9fb6-24efdc822394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:50.925682 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1a9de-7f92-4b23-9fb6-24efdc822394 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:50.926380 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1a9de-7f92-4b23-9fb6-24efdc822394 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:50.928371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1a9de-7f92-4b23-9fb6-24efdc822394 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:51.004611 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:51.011686 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1394/2794] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:34:50 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:34:51.015837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0ddc3-5521-4500-9707-0c012113c623 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:51.026388 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0ddc3-5521-4500-9707-0c012113c623 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:51.083967 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03770a76-e1b1-489a-b51d-cb19b85ff791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:51.084640 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03770a76-e1b1-489a-b51d-cb19b85ff791 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:51.084929 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03770a76-e1b1-489a-b51d-cb19b85ff791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:51.085208 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03770a76-e1b1-489a-b51d-cb19b85ff791 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:51.086421 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03770a76-e1b1-489a-b51d-cb19b85ff791 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:51.088388 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03770a76-e1b1-489a-b51d-cb19b85ff791 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:51.176197 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1401/2795] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:34:51 2022] POST /identity/v3/users => generated 282 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:34:51.180073 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1a9de-7f92-4b23-9fb6-24efdc822394 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:51.190158 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1a9de-7f92-4b23-9fb6-24efdc822394 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:51.195345 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589ccc9-4d37-4862-888a-c6c1f4239550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:51.196005 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589ccc9-4d37-4862-888a-c6c1f4239550 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:51.196296 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589ccc9-4d37-4862-888a-c6c1f4239550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:51.196578 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589ccc9-4d37-4862-888a-c6c1f4239550 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:51.197316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589ccc9-4d37-4862-888a-c6c1f4239550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:51.198953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589ccc9-4d37-4862-888a-c6c1f4239550 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:51.209307 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1395/2796] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:34:51.213775 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03770a76-e1b1-489a-b51d-cb19b85ff791 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:51.223892 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03770a76-e1b1-489a-b51d-cb19b85ff791 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:51.229177 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf39d36-3b73-419f-a46b-8c8f9ec61947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:51.229837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf39d36-3b73-419f-a46b-8c8f9ec61947 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:51.231660 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf39d36-3b73-419f-a46b-8c8f9ec61947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:51.231660 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf39d36-3b73-419f-a46b-8c8f9ec61947 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:51.231660 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf39d36-3b73-419f-a46b-8c8f9ec61947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:51.232855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf39d36-3b73-419f-a46b-8c8f9ec61947 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:51.243173 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1402/2797] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:34:51.247238 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589ccc9-4d37-4862-888a-c6c1f4239550 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:51.258088 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589ccc9-4d37-4862-888a-c6c1f4239550 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:51.262599 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148aa5cb-61c8-43f8-b5dc-1d94929263e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:51.263728 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148aa5cb-61c8-43f8-b5dc-1d94929263e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:51.264018 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148aa5cb-61c8-43f8-b5dc-1d94929263e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:51.264296 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148aa5cb-61c8-43f8-b5dc-1d94929263e0 admin admin] PATH_INFO: `/v3/projects/0bcf79bbfa2f4d619f897f6219cb2bf4/users/4102804ef0554e939721d47615681a66/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:51.279235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148aa5cb-61c8-43f8-b5dc-1d94929263e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bcf79bbfa2f4d619f897f6219cb2bf4, user_id=4102804ef0554e939721d47615681a66, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:51.281902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148aa5cb-61c8-43f8-b5dc-1d94929263e0 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:51.358809 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1396/2798] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:34:51 2022] PUT /identity/v3/projects/0bcf79bbfa2f4d619f897f6219cb2bf4/users/4102804ef0554e939721d47615681a66/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:34:51.369650 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f9558-77ad-475f-9843-a0a8b14ac30e None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:51.369949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f9558-77ad-475f-9843-a0a8b14ac30e None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:51.370236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f9558-77ad-475f-9843-a0a8b14ac30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:51.399756 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-aa8f9558-77ad-475f-9843-a0a8b14ac30e None None] MFA Rules not processed for user `4102804ef0554e939721d47615681a66`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:34:51.400954 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-aa8f9558-77ad-475f-9843-a0a8b14ac30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:51.463247 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1403/2799] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:34:51 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:34:51.467623 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148aa5cb-61c8-43f8-b5dc-1d94929263e0 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:51.527605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148aa5cb-61c8-43f8-b5dc-1d94929263e0 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:51.532285 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b4b1a-a8ed-4a13-b624-4749c7e0a0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:51.532938 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b4b1a-a8ed-4a13-b624-4749c7e0a0dd admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:51.533226 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b4b1a-a8ed-4a13-b624-4749c7e0a0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:51.533513 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b4b1a-a8ed-4a13-b624-4749c7e0a0dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:51.534209 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b4b1a-a8ed-4a13-b624-4749c7e0a0dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:51.536146 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b4b1a-a8ed-4a13-b624-4749c7e0a0dd admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:51.619502 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1397/2800] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:34:51 2022] POST /identity/v3/projects => generated 363 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:34:51.623435 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f9558-77ad-475f-9843-a0a8b14ac30e None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:51.634645 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f9558-77ad-475f-9843-a0a8b14ac30e None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:51.677408 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:51.693496 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835e2e5-3e4f-4f1a-9506-3b1dbb64d375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:51.694287 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835e2e5-3e4f-4f1a-9506-3b1dbb64d375 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:51.694580 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835e2e5-3e4f-4f1a-9506-3b1dbb64d375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:51.694862 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835e2e5-3e4f-4f1a-9506-3b1dbb64d375 admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:51.696159 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835e2e5-3e4f-4f1a-9506-3b1dbb64d375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:51.698150 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835e2e5-3e4f-4f1a-9506-3b1dbb64d375 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:51.813135 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1404/2801] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:34:51 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:34:51.817330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b4b1a-a8ed-4a13-b624-4749c7e0a0dd admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:51.828293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b4b1a-a8ed-4a13-b624-4749c7e0a0dd admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:51.832934 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00bf8b-0827-4925-88f0-48cd6c8d1d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:51.833605 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00bf8b-0827-4925-88f0-48cd6c8d1d70 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:51.833904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00bf8b-0827-4925-88f0-48cd6c8d1d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:51.834185 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00bf8b-0827-4925-88f0-48cd6c8d1d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:51.834944 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00bf8b-0827-4925-88f0-48cd6c8d1d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:51.836564 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00bf8b-0827-4925-88f0-48cd6c8d1d70 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:51.847004 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1398/2802] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:34:51.850992 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835e2e5-3e4f-4f1a-9506-3b1dbb64d375 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:51.860932 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835e2e5-3e4f-4f1a-9506-3b1dbb64d375 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:51.865559 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f3ea5-6286-43d6-980a-8450c16ff063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:51.866697 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f3ea5-6286-43d6-980a-8450c16ff063 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:51.867582 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f3ea5-6286-43d6-980a-8450c16ff063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:51.867862 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f3ea5-6286-43d6-980a-8450c16ff063 admin admin] PATH_INFO: `/v3/projects/01d175c799984ef0a02bca60b3273743/users/4fcc6034213440a7abfc0e30d5130a62/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:51.883564 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f3ea5-6286-43d6-980a-8450c16ff063 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01d175c799984ef0a02bca60b3273743, user_id=4fcc6034213440a7abfc0e30d5130a62, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:51.886223 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f3ea5-6286-43d6-980a-8450c16ff063 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:52.011293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1405/2803] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:34:51 2022] PUT /identity/v3/projects/01d175c799984ef0a02bca60b3273743/users/4fcc6034213440a7abfc0e30d5130a62/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:34:52.015315 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00bf8b-0827-4925-88f0-48cd6c8d1d70 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:52.025315 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00bf8b-0827-4925-88f0-48cd6c8d1d70 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:52.085176 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d977a-463e-4e7f-85cf-4ed24a088f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:52.085855 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d977a-463e-4e7f-85cf-4ed24a088f9c admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:52.086145 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d977a-463e-4e7f-85cf-4ed24a088f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:52.086428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d977a-463e-4e7f-85cf-4ed24a088f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:52.087195 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d977a-463e-4e7f-85cf-4ed24a088f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:52.088790 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d977a-463e-4e7f-85cf-4ed24a088f9c admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:52.099117 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1399/2804] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:34:52.108251 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f3ea5-6286-43d6-980a-8450c16ff063 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:52.119779 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f3ea5-6286-43d6-980a-8450c16ff063 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:52.124462 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce1010-cc5e-4950-8b6b-f06847ac3cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:52.125369 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce1010-cc5e-4950-8b6b-f06847ac3cce admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:52.125649 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce1010-cc5e-4950-8b6b-f06847ac3cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:52.125927 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce1010-cc5e-4950-8b6b-f06847ac3cce admin admin] PATH_INFO: `/v3/projects/01d175c799984ef0a02bca60b3273743/users/4fcc6034213440a7abfc0e30d5130a62/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:52.128023 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce1010-cc5e-4950-8b6b-f06847ac3cce admin admin] RBAC: Authorizing `identity:create_grant(project_id=01d175c799984ef0a02bca60b3273743, user_id=4fcc6034213440a7abfc0e30d5130a62, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:52.130857 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce1010-cc5e-4950-8b6b-f06847ac3cce admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:52.141272 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1406/2805] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:34:52 2022] PUT /identity/v3/projects/01d175c799984ef0a02bca60b3273743/users/4fcc6034213440a7abfc0e30d5130a62/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:34:52.152856 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f7382-02ac-4429-bab1-26ede487f08a None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:52.153240 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f7382-02ac-4429-bab1-26ede487f08a None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:52.153709 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f7382-02ac-4429-bab1-26ede487f08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:52.183592 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-353f7382-02ac-4429-bab1-26ede487f08a None None] MFA Rules not processed for user `4fcc6034213440a7abfc0e30d5130a62`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:34:52.184793 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-353f7382-02ac-4429-bab1-26ede487f08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:52.204997 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:52.252603 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1400/2806] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:34:52 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:34:52.263924 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce1010-cc5e-4950-8b6b-f06847ac3cce admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:52.265597 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-9cce1010-cc5e-4950-8b6b-f06847ac3cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:52.322424 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce1010-cc5e-4950-8b6b-f06847ac3cce admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:52.326041 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99894f69-9fe3-4c1c-b93b-3419fd48ddd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:52.326706 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99894f69-9fe3-4c1c-b93b-3419fd48ddd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:52.327016 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99894f69-9fe3-4c1c-b93b-3419fd48ddd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:52.327294 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99894f69-9fe3-4c1c-b93b-3419fd48ddd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:52.336219 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99894f69-9fe3-4c1c-b93b-3419fd48ddd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:52.337654 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99894f69-9fe3-4c1c-b93b-3419fd48ddd0 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:52.347105 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1407/2807] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:34:52 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:34:53.289431 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f7382-02ac-4429-bab1-26ede487f08a None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:53.351284 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f7382-02ac-4429-bab1-26ede487f08a None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:53.355974 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7d83a-43d2-415c-9c47-0a73100fbb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:53.356884 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7d83a-43d2-415c-9c47-0a73100fbb8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:53.357168 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7d83a-43d2-415c-9c47-0a73100fbb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:53.357451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7d83a-43d2-415c-9c47-0a73100fbb8e admin admin] PATH_INFO: `/v3/users/4102804ef0554e939721d47615681a66` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:53.358656 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7d83a-43d2-415c-9c47-0a73100fbb8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4102804ef0554e939721d47615681a66)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:53.360616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7d83a-43d2-415c-9c47-0a73100fbb8e admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:53.579215 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-8ca7d83a-43d2-415c-9c47-0a73100fbb8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4102804ef0554e939721d47615681a66'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:53.585997 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-8ca7d83a-43d2-415c-9c47-0a73100fbb8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4102804ef0554e939721d47615681a66'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:53.586663 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-8ca7d83a-43d2-415c-9c47-0a73100fbb8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4102804ef0554e939721d47615681a66'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:53.587290 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-8ca7d83a-43d2-415c-9c47-0a73100fbb8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4102804ef0554e939721d47615681a66'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:53.598836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-8ca7d83a-43d2-415c-9c47-0a73100fbb8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4102804ef0554e939721d47615681a66'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:53.718248 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1401/2808] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:34:53 2022] DELETE /identity/v3/users/4102804ef0554e939721d47615681a66 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:34:53.722739 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50141b17-9c77-439c-b880-31b1e011b54f None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:53.723270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50141b17-9c77-439c-b880-31b1e011b54f None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:53.723389 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50141b17-9c77-439c-b880-31b1e011b54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:53.741628 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-50141b17-9c77-439c-b880-31b1e011b54f None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:34:53.743493 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-50141b17-9c77-439c-b880-31b1e011b54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:53.805442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1408/2809] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:34:53.815868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7d83a-43d2-415c-9c47-0a73100fbb8e admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:53.817689 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-8ca7d83a-43d2-415c-9c47-0a73100fbb8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:53.858963 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:53.874282 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7d83a-43d2-415c-9c47-0a73100fbb8e admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:53.878062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38175e4-4419-40f1-aa58-ec049e9aeca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:53.878720 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38175e4-4419-40f1-aa58-ec049e9aeca2 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:53.879030 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38175e4-4419-40f1-aa58-ec049e9aeca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:53.886698 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38175e4-4419-40f1-aa58-ec049e9aeca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:53.888641 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38175e4-4419-40f1-aa58-ec049e9aeca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:53.890084 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38175e4-4419-40f1-aa58-ec049e9aeca2 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:53.900225 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1402/2810] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:34:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:34:54.368222 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50141b17-9c77-439c-b880-31b1e011b54f None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:54.369791 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-50141b17-9c77-439c-b880-31b1e011b54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:54.382629 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50141b17-9c77-439c-b880-31b1e011b54f None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:54.387965 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25148ed5-ea63-43a4-a43f-08df4d7b156e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:54.388838 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25148ed5-ea63-43a4-a43f-08df4d7b156e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:54.389313 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25148ed5-ea63-43a4-a43f-08df4d7b156e admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:54.389600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25148ed5-ea63-43a4-a43f-08df4d7b156e admin admin] PATH_INFO: `/v3/projects/0bcf79bbfa2f4d619f897f6219cb2bf4` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:54.391298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25148ed5-ea63-43a4-a43f-08df4d7b156e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bcf79bbfa2f4d619f897f6219cb2bf4)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:54.393278 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25148ed5-ea63-43a4-a43f-08df4d7b156e admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:54.493828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:54.505682 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-25148ed5-ea63-43a4-a43f-08df4d7b156e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bcf79bbfa2f4d619f897f6219cb2bf4', 'request_id': 'req-25148ed5-ea63-43a4-a43f-08df4d7b156e'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:54.512357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-25148ed5-ea63-43a4-a43f-08df4d7b156e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bcf79bbfa2f4d619f897f6219cb2bf4', 'request_id': 'req-25148ed5-ea63-43a4-a43f-08df4d7b156e'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:54.572673 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-25148ed5-ea63-43a4-a43f-08df4d7b156e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bcf79bbfa2f4d619f897f6219cb2bf4', 'request_id': 'req-25148ed5-ea63-43a4-a43f-08df4d7b156e'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:54.583734 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-25148ed5-ea63-43a4-a43f-08df4d7b156e admin admin] The token cache is being invalidate because project 0bcf79bbfa2f4d619f897f6219cb2bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:34:54.584021 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-25148ed5-ea63-43a4-a43f-08df4d7b156e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:54.584673 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-25148ed5-ea63-43a4-a43f-08df4d7b156e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:54.586237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1409/2811] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:34:54 2022] DELETE /identity/v3/projects/0bcf79bbfa2f4d619f897f6219cb2bf4 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:34:54.590766 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38175e4-4419-40f1-aa58-ec049e9aeca2 service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:54.601565 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38175e4-4419-40f1-aa58-ec049e9aeca2 service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:54.658344 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0791e48-fbbb-4116-aaa1-d1b3c68476f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:54.659272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0791e48-fbbb-4116-aaa1-d1b3c68476f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:54.659559 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0791e48-fbbb-4116-aaa1-d1b3c68476f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:54.659850 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0791e48-fbbb-4116-aaa1-d1b3c68476f6 admin admin] PATH_INFO: `/v3/users/4fcc6034213440a7abfc0e30d5130a62` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:54.661054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0791e48-fbbb-4116-aaa1-d1b3c68476f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fcc6034213440a7abfc0e30d5130a62)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:54.662982 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0791e48-fbbb-4116-aaa1-d1b3c68476f6 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:54.963316 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c0791e48-fbbb-4116-aaa1-d1b3c68476f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fcc6034213440a7abfc0e30d5130a62'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:54.970100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c0791e48-fbbb-4116-aaa1-d1b3c68476f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fcc6034213440a7abfc0e30d5130a62'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:54.970762 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c0791e48-fbbb-4116-aaa1-d1b3c68476f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fcc6034213440a7abfc0e30d5130a62'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:54.971403 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c0791e48-fbbb-4116-aaa1-d1b3c68476f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fcc6034213440a7abfc0e30d5130a62'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:54.983878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c0791e48-fbbb-4116-aaa1-d1b3c68476f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fcc6034213440a7abfc0e30d5130a62'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:55.112394 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1403/2812] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:34:54 2022] DELETE /identity/v3/users/4fcc6034213440a7abfc0e30d5130a62 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:34:55.499878 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25148ed5-ea63-43a4-a43f-08df4d7b156e admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:55.501977 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-25148ed5-ea63-43a4-a43f-08df4d7b156e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:55.518057 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25148ed5-ea63-43a4-a43f-08df4d7b156e admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:55.579512 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b18390-02da-44d1-9453-889629fb39f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:55.580380 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b18390-02da-44d1-9453-889629fb39f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:55.580668 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b18390-02da-44d1-9453-889629fb39f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:55.580944 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b18390-02da-44d1-9453-889629fb39f5 admin admin] PATH_INFO: `/v3/projects/01d175c799984ef0a02bca60b3273743` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:55.582148 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b18390-02da-44d1-9453-889629fb39f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01d175c799984ef0a02bca60b3273743)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:55.584099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b18390-02da-44d1-9453-889629fb39f5 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:55.654069 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0791e48-fbbb-4116-aaa1-d1b3c68476f6 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:55.655541 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-c0791e48-fbbb-4116-aaa1-d1b3c68476f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:55.669017 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0791e48-fbbb-4116-aaa1-d1b3c68476f6 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:55.673674 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35131f43-19b1-4e7e-bf6f-4e7d3c7e4c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:55.674557 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35131f43-19b1-4e7e-bf6f-4e7d3c7e4c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:55.674850 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35131f43-19b1-4e7e-bf6f-4e7d3c7e4c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:55.675169 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35131f43-19b1-4e7e-bf6f-4e7d3c7e4c12 admin admin] PATH_INFO: `/v3/users/b1ec74f16cf147c0a799864bbe280ac8` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:55.676369 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35131f43-19b1-4e7e-bf6f-4e7d3c7e4c12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1ec74f16cf147c0a799864bbe280ac8)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:55.678287 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35131f43-19b1-4e7e-bf6f-4e7d3c7e4c12 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:55.709633 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-87b18390-02da-44d1-9453-889629fb39f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01d175c799984ef0a02bca60b3273743', 'request_id': 'req-87b18390-02da-44d1-9453-889629fb39f5'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:55.716276 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-87b18390-02da-44d1-9453-889629fb39f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01d175c799984ef0a02bca60b3273743', 'request_id': 'req-87b18390-02da-44d1-9453-889629fb39f5'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:55.858253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-87b18390-02da-44d1-9453-889629fb39f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01d175c799984ef0a02bca60b3273743', 'request_id': 'req-87b18390-02da-44d1-9453-889629fb39f5'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:55.870600 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-87b18390-02da-44d1-9453-889629fb39f5 admin admin] The token cache is being invalidate because project 01d175c799984ef0a02bca60b3273743 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:34:55.870893 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-87b18390-02da-44d1-9453-889629fb39f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:55.871574 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-87b18390-02da-44d1-9453-889629fb39f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:55.873264 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1410/2813] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:34:55 2022] DELETE /identity/v3/projects/01d175c799984ef0a02bca60b3273743 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:34:55.877848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bceca1a-7a48-42c7-a032-d9d2443a6f08 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:55.878144 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bceca1a-7a48-42c7-a032-d9d2443a6f08 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:55.878430 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bceca1a-7a48-42c7-a032-d9d2443a6f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:55.896953 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-3bceca1a-7a48-42c7-a032-d9d2443a6f08 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:34:55.898156 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-3bceca1a-7a48-42c7-a032-d9d2443a6f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:55.942679 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:55.957372 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1411/2814] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:34:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:34:55.965687 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34eec11-398e-4e18-b429-316f495f6c0f None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:55.965915 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34eec11-398e-4e18-b429-316f495f6c0f None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:55.966196 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34eec11-398e-4e18-b429-316f495f6c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:55.984931 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-a34eec11-398e-4e18-b429-316f495f6c0f None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:34:55.986130 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a34eec11-398e-4e18-b429-316f495f6c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:55.986798 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-35131f43-19b1-4e7e-bf6f-4e7d3c7e4c12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1ec74f16cf147c0a799864bbe280ac8'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:55.993185 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:56.002190 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-35131f43-19b1-4e7e-bf6f-4e7d3c7e4c12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1ec74f16cf147c0a799864bbe280ac8'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:56.002526 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1412/2815] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:34:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:34:56.002641 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-35131f43-19b1-4e7e-bf6f-4e7d3c7e4c12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1ec74f16cf147c0a799864bbe280ac8'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:56.003288 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34eec11-398e-4e18-b429-316f495f6c0f None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:56.003452 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-35131f43-19b1-4e7e-bf6f-4e7d3c7e4c12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1ec74f16cf147c0a799864bbe280ac8'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:56.015255 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34eec11-398e-4e18-b429-316f495f6c0f None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:56.016835 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-35131f43-19b1-4e7e-bf6f-4e7d3c7e4c12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1ec74f16cf147c0a799864bbe280ac8'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:56.020602 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01707bbd-81a2-4b37-bcf4-192c661326af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:56.021268 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01707bbd-81a2-4b37-bcf4-192c661326af admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:56.021568 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01707bbd-81a2-4b37-bcf4-192c661326af admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:56.021830 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01707bbd-81a2-4b37-bcf4-192c661326af admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:56.022574 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01707bbd-81a2-4b37-bcf4-192c661326af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:34:56.022884 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01707bbd-81a2-4b37-bcf4-192c661326af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:56.024586 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01707bbd-81a2-4b37-bcf4-192c661326af admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:56.038831 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1413/2816] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 09:34:56.043109 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01707bbd-81a2-4b37-bcf4-192c661326af admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:56.044834 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-01707bbd-81a2-4b37-bcf4-192c661326af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:56.057711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01707bbd-81a2-4b37-bcf4-192c661326af admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:56.062688 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3fd568-4115-4b03-923c-68aecfd4f783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:56.063373 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3fd568-4115-4b03-923c-68aecfd4f783 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:56.063661 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3fd568-4115-4b03-923c-68aecfd4f783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:56.063937 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3fd568-4115-4b03-923c-68aecfd4f783 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:56.064632 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3fd568-4115-4b03-923c-68aecfd4f783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:56.066688 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3fd568-4115-4b03-923c-68aecfd4f783 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:56.095636 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1404/2817] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:34:55 2022] DELETE /identity/v3/users/b1ec74f16cf147c0a799864bbe280ac8 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:34:56.100029 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d694b-1ae0-4ec0-bd2e-d75d04c4538a None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:56.100333 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d694b-1ae0-4ec0-bd2e-d75d04c4538a None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:56.100650 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d694b-1ae0-4ec0-bd2e-d75d04c4538a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:56.118172 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-a25d694b-1ae0-4ec0-bd2e-d75d04c4538a None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:34:56.119944 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-a25d694b-1ae0-4ec0-bd2e-d75d04c4538a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:56.176262 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1414/2818] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:34:56 2022] POST /identity/v3/projects => generated 381 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:34:56.176725 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3fd568-4115-4b03-923c-68aecfd4f783 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:56.190600 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1405/2819] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:34:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:34:56.201843 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d694b-1ae0-4ec0-bd2e-d75d04c4538a None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:56.203367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-a25d694b-1ae0-4ec0-bd2e-d75d04c4538a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:56.245094 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3fd568-4115-4b03-923c-68aecfd4f783 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:56.249872 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e643a-2318-4568-8138-7204200046ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:56.250692 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e643a-2318-4568-8138-7204200046ba admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:56.250828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e643a-2318-4568-8138-7204200046ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:56.251155 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e643a-2318-4568-8138-7204200046ba admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:56.251843 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e643a-2318-4568-8138-7204200046ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:56.253822 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e643a-2318-4568-8138-7204200046ba admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:56.265148 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d694b-1ae0-4ec0-bd2e-d75d04c4538a None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:56.318595 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd14372-97d0-45b3-98f0-ad49d8400815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:56.319293 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd14372-97d0-45b3-98f0-ad49d8400815 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:56.319579 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd14372-97d0-45b3-98f0-ad49d8400815 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:56.319857 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd14372-97d0-45b3-98f0-ad49d8400815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:56.329520 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd14372-97d0-45b3-98f0-ad49d8400815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:56.331042 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd14372-97d0-45b3-98f0-ad49d8400815 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:56.340327 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1406/2820] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:34:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:34:56.348098 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1415/2821] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:34:56 2022] POST /identity/v3/users => generated 291 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:34:56.351682 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd14372-97d0-45b3-98f0-ad49d8400815 service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:56.362695 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd14372-97d0-45b3-98f0-ad49d8400815 service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:56.368822 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cba403-51f2-41c5-a957-1149a83f4ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:56.369497 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cba403-51f2-41c5-a957-1149a83f4ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:56.370137 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cba403-51f2-41c5-a957-1149a83f4ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:56.370137 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cba403-51f2-41c5-a957-1149a83f4ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:56.370853 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cba403-51f2-41c5-a957-1149a83f4ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:56.372612 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cba403-51f2-41c5-a957-1149a83f4ad8 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:56.382843 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1407/2822] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:34:56.387437 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e643a-2318-4568-8138-7204200046ba admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:56.397482 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e643a-2318-4568-8138-7204200046ba admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:56.403925 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a54ea4-465f-4ed0-9a6f-7510be23bbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:56.404602 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a54ea4-465f-4ed0-9a6f-7510be23bbba admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:56.404891 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a54ea4-465f-4ed0-9a6f-7510be23bbba admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:56.405166 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a54ea4-465f-4ed0-9a6f-7510be23bbba admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:56.405950 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a54ea4-465f-4ed0-9a6f-7510be23bbba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:56.407599 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a54ea4-465f-4ed0-9a6f-7510be23bbba admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:56.419117 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1416/2823] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:34:56.423596 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cba403-51f2-41c5-a957-1149a83f4ad8 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:56.435275 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cba403-51f2-41c5-a957-1149a83f4ad8 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:56.468739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6dd17d-cc42-4a03-95ac-211ba513eabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:56.468739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6dd17d-cc42-4a03-95ac-211ba513eabc admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:56.468739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6dd17d-cc42-4a03-95ac-211ba513eabc admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:56.468739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6dd17d-cc42-4a03-95ac-211ba513eabc admin admin] PATH_INFO: `/v3/projects/6b83b46972d64a12b6944a3c32c6996e/users/852acc1b288f47a28f6e26aac8c67696/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:56.469223 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6dd17d-cc42-4a03-95ac-211ba513eabc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b83b46972d64a12b6944a3c32c6996e, user_id=852acc1b288f47a28f6e26aac8c67696, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:56.470805 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6dd17d-cc42-4a03-95ac-211ba513eabc admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:56.721753 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:56.729317 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1408/2824] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:34:56 2022] PUT /identity/v3/projects/6b83b46972d64a12b6944a3c32c6996e/users/852acc1b288f47a28f6e26aac8c67696/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:34:56.739612 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e07030-c96f-4eb4-a718-9008d1ba2258 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:56.740078 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e07030-c96f-4eb4-a718-9008d1ba2258 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:56.740187 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e07030-c96f-4eb4-a718-9008d1ba2258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:56.768686 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-a9e07030-c96f-4eb4-a718-9008d1ba2258 None None] MFA Rules not processed for user `852acc1b288f47a28f6e26aac8c67696`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:34:56.773586 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a9e07030-c96f-4eb4-a718-9008d1ba2258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:56.836536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1417/2825] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:34:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:34:56.840776 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6dd17d-cc42-4a03-95ac-211ba513eabc admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:56.904111 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6dd17d-cc42-4a03-95ac-211ba513eabc admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:56.908784 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4d8e5-0c0e-407a-91ce-3d20756d3666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:56.913639 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4d8e5-0c0e-407a-91ce-3d20756d3666 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:56.913639 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4d8e5-0c0e-407a-91ce-3d20756d3666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:56.913639 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4d8e5-0c0e-407a-91ce-3d20756d3666 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:56.913639 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4d8e5-0c0e-407a-91ce-3d20756d3666 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:56.913639 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4d8e5-0c0e-407a-91ce-3d20756d3666 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:56.942522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e07030-c96f-4eb4-a718-9008d1ba2258 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:56.942522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a9e07030-c96f-4eb4-a718-9008d1ba2258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:56.957315 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e07030-c96f-4eb4-a718-9008d1ba2258 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:56.963683 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb4e12-f93d-4e91-a7cb-cc9c51467520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:56.968658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb4e12-f93d-4e91-a7cb-cc9c51467520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:56.968658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb4e12-f93d-4e91-a7cb-cc9c51467520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:56.968658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb4e12-f93d-4e91-a7cb-cc9c51467520 admin admin] PATH_INFO: `/v3/projects/05bdf101dc8c4d22b770d3565fc5d2ce` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:56.968658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb4e12-f93d-4e91-a7cb-cc9c51467520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05bdf101dc8c4d22b770d3565fc5d2ce)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:56.968658 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb4e12-f93d-4e91-a7cb-cc9c51467520 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:56.981911 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:57.026012 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1409/2826] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:34:56 2022] POST /identity/v3/projects => generated 381 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:34:57.029637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4d8e5-0c0e-407a-91ce-3d20756d3666 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:57.090677 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4d8e5-0c0e-407a-91ce-3d20756d3666 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:57.096704 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ed20d-23ae-4fc5-8e75-3d97292f2868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:57.100827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ed20d-23ae-4fc5-8e75-3d97292f2868 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:57.100827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ed20d-23ae-4fc5-8e75-3d97292f2868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:57.100827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ed20d-23ae-4fc5-8e75-3d97292f2868 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:57.100827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ed20d-23ae-4fc5-8e75-3d97292f2868 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:57.100827 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ed20d-23ae-4fc5-8e75-3d97292f2868 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:57.111784 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-72bb4e12-f93d-4e91-a7cb-cc9c51467520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05bdf101dc8c4d22b770d3565fc5d2ce', 'request_id': 'req-72bb4e12-f93d-4e91-a7cb-cc9c51467520'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:57.118628 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-72bb4e12-f93d-4e91-a7cb-cc9c51467520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05bdf101dc8c4d22b770d3565fc5d2ce', 'request_id': 'req-72bb4e12-f93d-4e91-a7cb-cc9c51467520'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:57.187531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1410/2827] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:34:57 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 09:34:57.192639 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ed20d-23ae-4fc5-8e75-3d97292f2868 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:57.200700 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ed20d-23ae-4fc5-8e75-3d97292f2868 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:57.213767 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-72bb4e12-f93d-4e91-a7cb-cc9c51467520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05bdf101dc8c4d22b770d3565fc5d2ce', 'request_id': 'req-72bb4e12-f93d-4e91-a7cb-cc9c51467520'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:57.224579 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-72bb4e12-f93d-4e91-a7cb-cc9c51467520 admin admin] The token cache is being invalidate because project 05bdf101dc8c4d22b770d3565fc5d2ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:34:57.224868 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-72bb4e12-f93d-4e91-a7cb-cc9c51467520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:57.225597 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-72bb4e12-f93d-4e91-a7cb-cc9c51467520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:57.227202 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1418/2828] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:34:56 2022] DELETE /identity/v3/projects/05bdf101dc8c4d22b770d3565fc5d2ce => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:34:57.261910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39fe6b-bccc-4cab-aaaa-f8bf2a1128c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:57.266237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39fe6b-bccc-4cab-aaaa-f8bf2a1128c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:57.266237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39fe6b-bccc-4cab-aaaa-f8bf2a1128c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:57.266237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39fe6b-bccc-4cab-aaaa-f8bf2a1128c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:57.266237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39fe6b-bccc-4cab-aaaa-f8bf2a1128c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:57.266237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39fe6b-bccc-4cab-aaaa-f8bf2a1128c6 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:57.280689 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1411/2829] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:34:57.281945 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb4e12-f93d-4e91-a7cb-cc9c51467520 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:57.283515 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-72bb4e12-f93d-4e91-a7cb-cc9c51467520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:57.297299 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb4e12-f93d-4e91-a7cb-cc9c51467520 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:57.302448 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f47d60-39d2-410e-9b8a-69c1306668e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:57.303530 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f47d60-39d2-410e-9b8a-69c1306668e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:57.303885 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f47d60-39d2-410e-9b8a-69c1306668e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:57.304232 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f47d60-39d2-410e-9b8a-69c1306668e7 admin admin] PATH_INFO: `/v3/projects/8468f1c844cb4872bac509254899f6ae/users/3370f6e850e641a4847a40eacfdcb199/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:57.322895 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f47d60-39d2-410e-9b8a-69c1306668e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8468f1c844cb4872bac509254899f6ae, user_id=3370f6e850e641a4847a40eacfdcb199, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:57.326621 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f47d60-39d2-410e-9b8a-69c1306668e7 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:57.422343 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1419/2830] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:34:57 2022] PUT /identity/v3/projects/8468f1c844cb4872bac509254899f6ae/users/3370f6e850e641a4847a40eacfdcb199/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:34:57.429690 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39fe6b-bccc-4cab-aaaa-f8bf2a1128c6 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:57.440415 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39fe6b-bccc-4cab-aaaa-f8bf2a1128c6 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:57.444608 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:57.517616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6127e37-8002-4d19-ac7d-a780d29beb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:57.518294 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6127e37-8002-4d19-ac7d-a780d29beb67 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:57.518585 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6127e37-8002-4d19-ac7d-a780d29beb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:57.518868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6127e37-8002-4d19-ac7d-a780d29beb67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:57.519638 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6127e37-8002-4d19-ac7d-a780d29beb67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:57.521240 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6127e37-8002-4d19-ac7d-a780d29beb67 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:57.531872 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1412/2831] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:34:57.536143 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f47d60-39d2-410e-9b8a-69c1306668e7 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:57.546839 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f47d60-39d2-410e-9b8a-69c1306668e7 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:57.552104 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeff4de-cf82-4fc6-8b64-70580cfd76d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:57.553099 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeff4de-cf82-4fc6-8b64-70580cfd76d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:57.553388 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeff4de-cf82-4fc6-8b64-70580cfd76d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:57.553756 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeff4de-cf82-4fc6-8b64-70580cfd76d9 admin admin] PATH_INFO: `/v3/projects/8468f1c844cb4872bac509254899f6ae/users/3370f6e850e641a4847a40eacfdcb199/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:57.555886 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeff4de-cf82-4fc6-8b64-70580cfd76d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8468f1c844cb4872bac509254899f6ae, user_id=3370f6e850e641a4847a40eacfdcb199, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:57.558969 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeff4de-cf82-4fc6-8b64-70580cfd76d9 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:57.569866 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1420/2832] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:34:57 2022] PUT /identity/v3/projects/8468f1c844cb4872bac509254899f6ae/users/3370f6e850e641a4847a40eacfdcb199/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:34:57.581619 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebfd49-adb3-4b90-8089-93b92e50d01c None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:57.581910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebfd49-adb3-4b90-8089-93b92e50d01c None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:57.582187 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebfd49-adb3-4b90-8089-93b92e50d01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:57.614038 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-b2ebfd49-adb3-4b90-8089-93b92e50d01c None None] MFA Rules not processed for user `3370f6e850e641a4847a40eacfdcb199`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:34:57.615424 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b2ebfd49-adb3-4b90-8089-93b92e50d01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:57.695480 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1413/2833] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:34:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:34:57.711024 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeff4de-cf82-4fc6-8b64-70580cfd76d9 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:57.713010 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-beeff4de-cf82-4fc6-8b64-70580cfd76d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:57.740294 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ebfd49-adb3-4b90-8089-93b92e50d01c None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:57.742004 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b2ebfd49-adb3-4b90-8089-93b92e50d01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:57.775446 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeff4de-cf82-4fc6-8b64-70580cfd76d9 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:57.779401 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635fdb9-6eac-47d2-9de2-de0719d29cf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:57.780080 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635fdb9-6eac-47d2-9de2-de0719d29cf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:57.780366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635fdb9-6eac-47d2-9de2-de0719d29cf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:57.780648 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635fdb9-6eac-47d2-9de2-de0719d29cf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:57.790494 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635fdb9-6eac-47d2-9de2-de0719d29cf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:57.792524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635fdb9-6eac-47d2-9de2-de0719d29cf1 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:57.802587 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1421/2834] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:34:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:34:57.825194 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ebfd49-adb3-4b90-8089-93b92e50d01c None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:57.832009 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e756cc8-9175-40eb-af4b-dacff1891023 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff025cb706d243778468ae4570374690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:57.832816 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e756cc8-9175-40eb-af4b-dacff1891023 service nova] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:57.833174 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e756cc8-9175-40eb-af4b-dacff1891023 service nova] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:57.833529 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e756cc8-9175-40eb-af4b-dacff1891023 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:57.837165 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-7e756cc8-9175-40eb-af4b-dacff1891023 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:34:57.858771 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e756cc8-9175-40eb-af4b-dacff1891023 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:57.860485 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e756cc8-9175-40eb-af4b-dacff1891023 service nova] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:57.873800 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1414/2835] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:34:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:34:58.907684 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635fdb9-6eac-47d2-9de2-de0719d29cf1 service cinder] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:58.916883 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635fdb9-6eac-47d2-9de2-de0719d29cf1 service cinder] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:58.921500 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476bf8a-cb6b-4526-b589-bb10f9c4fcfc tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3370f6e850e641a4847a40eacfdcb199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8468f1c844cb4872bac509254899f6ae', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:58.922155 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476bf8a-cb6b-4526-b589-bb10f9c4fcfc tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:58.922447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476bf8a-cb6b-4526-b589-bb10f9c4fcfc tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:58.922731 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476bf8a-cb6b-4526-b589-bb10f9c4fcfc tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:58.923493 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1476bf8a-cb6b-4526-b589-bb10f9c4fcfc tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:34:58.923839 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1476bf8a-cb6b-4526-b589-bb10f9c4fcfc tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:58.925447 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1476bf8a-cb6b-4526-b589-bb10f9c4fcfc tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:58.937008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1422/2836] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:34:58.941006 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e756cc8-9175-40eb-af4b-dacff1891023 service nova] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:58.952199 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e756cc8-9175-40eb-af4b-dacff1891023 service nova] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:58.956688 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18f276-bb10-4c76-902c-a9c7dc296fbc tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3370f6e850e641a4847a40eacfdcb199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8468f1c844cb4872bac509254899f6ae', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:58.957333 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18f276-bb10-4c76-902c-a9c7dc296fbc tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:58.957629 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18f276-bb10-4c76-902c-a9c7dc296fbc tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:58.957910 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18f276-bb10-4c76-902c-a9c7dc296fbc tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:58.958611 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18f276-bb10-4c76-902c-a9c7dc296fbc tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:58.960537 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18f276-bb10-4c76-902c-a9c7dc296fbc tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:58.971364 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=100 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:59.056221 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1415/2837] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:34:58 2022] POST /identity/v3/projects => generated 353 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:34:59.060461 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476bf8a-cb6b-4526-b589-bb10f9c4fcfc tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:59.070758 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476bf8a-cb6b-4526-b589-bb10f9c4fcfc tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:59.119527 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:34:59.127820 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e22ff-08e0-437a-bc12-243eb20a3cf5 tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3370f6e850e641a4847a40eacfdcb199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8468f1c844cb4872bac509254899f6ae', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:59.128606 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e22ff-08e0-437a-bc12-243eb20a3cf5 tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:59.128907 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e22ff-08e0-437a-bc12-243eb20a3cf5 tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:59.129192 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e22ff-08e0-437a-bc12-243eb20a3cf5 tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:59.129926 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e22ff-08e0-437a-bc12-243eb20a3cf5 tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:34:59.130239 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e22ff-08e0-437a-bc12-243eb20a3cf5 tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:59.131879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e22ff-08e0-437a-bc12-243eb20a3cf5 tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:59.144205 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1423/2838] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:34:59.180609 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18f276-bb10-4c76-902c-a9c7dc296fbc tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:34:59.191841 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18f276-bb10-4c76-902c-a9c7dc296fbc tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:34:59.197026 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d314a414-7cbc-4395-8a86-18418b7e096b tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3370f6e850e641a4847a40eacfdcb199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8468f1c844cb4872bac509254899f6ae', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:34:59.197890 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314a414-7cbc-4395-8a86-18418b7e096b tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:34:59.198181 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314a414-7cbc-4395-8a86-18418b7e096b tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:34:59.198459 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314a414-7cbc-4395-8a86-18418b7e096b tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] PATH_INFO: `/v3/projects/606cfa8b743d44b19405dacc79b63d20` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:34:59.199849 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d314a414-7cbc-4395-8a86-18418b7e096b tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] RBAC: Authorizing `identity:delete_project(project_id=606cfa8b743d44b19405dacc79b63d20)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:34:59.201750 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d314a414-7cbc-4395-8a86-18418b7e096b tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:34:59.310404 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-d314a414-7cbc-4395-8a86-18418b7e096b tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '606cfa8b743d44b19405dacc79b63d20', 'request_id': 'req-d314a414-7cbc-4395-8a86-18418b7e096b'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:59.316644 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-d314a414-7cbc-4395-8a86-18418b7e096b tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '606cfa8b743d44b19405dacc79b63d20', 'request_id': 'req-d314a414-7cbc-4395-8a86-18418b7e096b'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:59.323101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-d314a414-7cbc-4395-8a86-18418b7e096b tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '606cfa8b743d44b19405dacc79b63d20', 'request_id': 'req-d314a414-7cbc-4395-8a86-18418b7e096b'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:59.440878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-d314a414-7cbc-4395-8a86-18418b7e096b tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] The token cache is being invalidate because project 606cfa8b743d44b19405dacc79b63d20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:34:59.441208 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-d314a414-7cbc-4395-8a86-18418b7e096b tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:59.441801 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-d314a414-7cbc-4395-8a86-18418b7e096b tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:34:59.443475 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1416/2839] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:34:59 2022] DELETE /identity/v3/projects/606cfa8b743d44b19405dacc79b63d20 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:35:00.014869 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e22ff-08e0-437a-bc12-243eb20a3cf5 tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:00.016692 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-bc8e22ff-08e0-437a-bc12-243eb20a3cf5 tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:00.086848 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e22ff-08e0-437a-bc12-243eb20a3cf5 tempest-VolumeQuotaClassesTest-1995019258 tempest-VolumeQuotaClassesTest-1995019258-project-admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:00.144436 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf9721-6580-44be-86bb-14c270965a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:00.145313 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf9721-6580-44be-86bb-14c270965a57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:00.145604 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf9721-6580-44be-86bb-14c270965a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:00.145883 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf9721-6580-44be-86bb-14c270965a57 admin admin] PATH_INFO: `/v3/users/852acc1b288f47a28f6e26aac8c67696` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:00.147197 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf9721-6580-44be-86bb-14c270965a57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=852acc1b288f47a28f6e26aac8c67696)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:00.149146 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf9721-6580-44be-86bb-14c270965a57 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:00.350416 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2cdf9721-6580-44be-86bb-14c270965a57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '852acc1b288f47a28f6e26aac8c67696'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:00.357233 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2cdf9721-6580-44be-86bb-14c270965a57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '852acc1b288f47a28f6e26aac8c67696'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:00.357899 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2cdf9721-6580-44be-86bb-14c270965a57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '852acc1b288f47a28f6e26aac8c67696'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:00.358517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2cdf9721-6580-44be-86bb-14c270965a57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '852acc1b288f47a28f6e26aac8c67696'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:00.370665 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2cdf9721-6580-44be-86bb-14c270965a57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '852acc1b288f47a28f6e26aac8c67696'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:00.479514 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1424/2840] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:35:00 2022] DELETE /identity/v3/users/852acc1b288f47a28f6e26aac8c67696 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:35:00.484787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0eb10-c2ca-4cc9-b00e-d94efa70c0ae None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:00.484787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0eb10-c2ca-4cc9-b00e-d94efa70c0ae None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:00.484787 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0eb10-c2ca-4cc9-b00e-d94efa70c0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:00.502296 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-73e0eb10-c2ca-4cc9-b00e-d94efa70c0ae None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:35:00.503515 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-73e0eb10-c2ca-4cc9-b00e-d94efa70c0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:00.566723 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1417/2841] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:35:00.577047 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf9721-6580-44be-86bb-14c270965a57 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:00.580538 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-2cdf9721-6580-44be-86bb-14c270965a57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:00.593686 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:35:00.640055 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf9721-6580-44be-86bb-14c270965a57 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:00.643588 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d3096-6e09-4e80-abfa-7b6006b0b2fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:00.644238 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d3096-6e09-4e80-abfa-7b6006b0b2fd service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:00.644524 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d3096-6e09-4e80-abfa-7b6006b0b2fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:00.644804 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d3096-6e09-4e80-abfa-7b6006b0b2fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:00.654347 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d3096-6e09-4e80-abfa-7b6006b0b2fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:00.655907 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d3096-6e09-4e80-abfa-7b6006b0b2fd service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:00.665690 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1425/2842] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:35:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:35:01.088456 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0eb10-c2ca-4cc9-b00e-d94efa70c0ae None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:01.089945 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-73e0eb10-c2ca-4cc9-b00e-d94efa70c0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:01.102647 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0eb10-c2ca-4cc9-b00e-d94efa70c0ae None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:01.109347 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da461aac-e2a9-4bef-987f-29f90962b7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:01.110243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da461aac-e2a9-4bef-987f-29f90962b7f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:01.110531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da461aac-e2a9-4bef-987f-29f90962b7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:01.110808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da461aac-e2a9-4bef-987f-29f90962b7f5 admin admin] PATH_INFO: `/v3/projects/6b83b46972d64a12b6944a3c32c6996e` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:01.112182 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da461aac-e2a9-4bef-987f-29f90962b7f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b83b46972d64a12b6944a3c32c6996e)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:01.114194 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da461aac-e2a9-4bef-987f-29f90962b7f5 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:01.259772 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-da461aac-e2a9-4bef-987f-29f90962b7f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b83b46972d64a12b6944a3c32c6996e', 'request_id': 'req-da461aac-e2a9-4bef-987f-29f90962b7f5'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:01.266265 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-da461aac-e2a9-4bef-987f-29f90962b7f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b83b46972d64a12b6944a3c32c6996e', 'request_id': 'req-da461aac-e2a9-4bef-987f-29f90962b7f5'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:01.272127 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-da461aac-e2a9-4bef-987f-29f90962b7f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b83b46972d64a12b6944a3c32c6996e', 'request_id': 'req-da461aac-e2a9-4bef-987f-29f90962b7f5'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:01.382300 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=83,keystone:INSERT=3,keystone:DELETE=14 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:35:01.388756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-da461aac-e2a9-4bef-987f-29f90962b7f5 admin admin] The token cache is being invalidate because project 6b83b46972d64a12b6944a3c32c6996e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:35:01.388756 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-da461aac-e2a9-4bef-987f-29f90962b7f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:01.388987 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-da461aac-e2a9-4bef-987f-29f90962b7f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:01.390579 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1418/2843] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:35:01 2022] DELETE /identity/v3/projects/6b83b46972d64a12b6944a3c32c6996e => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:35:01.394750 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d3096-6e09-4e80-abfa-7b6006b0b2fd service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:01.405522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d3096-6e09-4e80-abfa-7b6006b0b2fd service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:01.464478 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3953a4-e99c-4691-9301-1968509aed1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:01.465360 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3953a4-e99c-4691-9301-1968509aed1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:01.465648 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3953a4-e99c-4691-9301-1968509aed1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:01.465929 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3953a4-e99c-4691-9301-1968509aed1c admin admin] PATH_INFO: `/v3/users/3370f6e850e641a4847a40eacfdcb199` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:01.467167 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3953a4-e99c-4691-9301-1968509aed1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3370f6e850e641a4847a40eacfdcb199)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:01.469114 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3953a4-e99c-4691-9301-1968509aed1c admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:01.682465 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-bb3953a4-e99c-4691-9301-1968509aed1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3370f6e850e641a4847a40eacfdcb199'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:01.689279 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-bb3953a4-e99c-4691-9301-1968509aed1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3370f6e850e641a4847a40eacfdcb199'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:01.689966 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-bb3953a4-e99c-4691-9301-1968509aed1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3370f6e850e641a4847a40eacfdcb199'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:01.690576 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-bb3953a4-e99c-4691-9301-1968509aed1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3370f6e850e641a4847a40eacfdcb199'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:01.703164 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-bb3953a4-e99c-4691-9301-1968509aed1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3370f6e850e641a4847a40eacfdcb199'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:01.804639 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1426/2844] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:35:01 2022] DELETE /identity/v3/users/3370f6e850e641a4847a40eacfdcb199 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:35:02.591972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da461aac-e2a9-4bef-987f-29f90962b7f5 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:02.593444 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-da461aac-e2a9-4bef-987f-29f90962b7f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:02.608929 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da461aac-e2a9-4bef-987f-29f90962b7f5 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:02.671736 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f5a83-b4f0-4f3d-a09b-06fd45ef153e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:02.674408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f5a83-b4f0-4f3d-a09b-06fd45ef153e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:02.674408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f5a83-b4f0-4f3d-a09b-06fd45ef153e admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:02.674408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f5a83-b4f0-4f3d-a09b-06fd45ef153e admin admin] PATH_INFO: `/v3/projects/8468f1c844cb4872bac509254899f6ae` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:02.674408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f5a83-b4f0-4f3d-a09b-06fd45ef153e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8468f1c844cb4872bac509254899f6ae)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:02.676512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f5a83-b4f0-4f3d-a09b-06fd45ef153e admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:02.829400 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c06f5a83-b4f0-4f3d-a09b-06fd45ef153e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8468f1c844cb4872bac509254899f6ae', 'request_id': 'req-c06f5a83-b4f0-4f3d-a09b-06fd45ef153e'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:02.836679 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c06f5a83-b4f0-4f3d-a09b-06fd45ef153e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8468f1c844cb4872bac509254899f6ae', 'request_id': 'req-c06f5a83-b4f0-4f3d-a09b-06fd45ef153e'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:02.843908 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c06f5a83-b4f0-4f3d-a09b-06fd45ef153e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8468f1c844cb4872bac509254899f6ae', 'request_id': 'req-c06f5a83-b4f0-4f3d-a09b-06fd45ef153e'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:02.955144 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c06f5a83-b4f0-4f3d-a09b-06fd45ef153e admin admin] The token cache is being invalidate because project 8468f1c844cb4872bac509254899f6ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80655) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:35:02.955666 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c06f5a83-b4f0-4f3d-a09b-06fd45ef153e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:02.956325 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-c06f5a83-b4f0-4f3d-a09b-06fd45ef153e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:02.961589 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1419/2845] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:35:02 2022] DELETE /identity/v3/projects/8468f1c844cb4872bac509254899f6ae => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:35:02.963994 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee8f5c-e348-485f-bb3f-81731b92444c None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:02.964477 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee8f5c-e348-485f-bb3f-81731b92444c None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:02.964596 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee8f5c-e348-485f-bb3f-81731b92444c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:02.983357 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-e6ee8f5c-e348-485f-bb3f-81731b92444c None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:35:02.984601 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-e6ee8f5c-e348-485f-bb3f-81731b92444c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:03.018479 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:35:03.048953 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1427/2846] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:35:03.056739 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e46d0-57c5-49a5-9f27-70b79b0927d0 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:03.057035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e46d0-57c5-49a5-9f27-70b79b0927d0 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:03.057371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e46d0-57c5-49a5-9f27-70b79b0927d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:03.078685 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-4f1e46d0-57c5-49a5-9f27-70b79b0927d0 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:35:03.080002 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-4f1e46d0-57c5-49a5-9f27-70b79b0927d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:03.089402 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1420/2847] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:35:03.093693 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee8f5c-e348-485f-bb3f-81731b92444c None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:03.103722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee8f5c-e348-485f-bb3f-81731b92444c None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:03.109265 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31b7e2-5d11-4862-8a1f-93bac401e9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:03.109946 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31b7e2-5d11-4862-8a1f-93bac401e9f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:03.110240 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31b7e2-5d11-4862-8a1f-93bac401e9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:03.110522 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31b7e2-5d11-4862-8a1f-93bac401e9f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:03.111721 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b31b7e2-5d11-4862-8a1f-93bac401e9f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:35:03.111721 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b31b7e2-5d11-4862-8a1f-93bac401e9f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:03.113332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b31b7e2-5d11-4862-8a1f-93bac401e9f4 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:03.127323 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1428/2848] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:35:03.131311 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e46d0-57c5-49a5-9f27-70b79b0927d0 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:03.141153 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e46d0-57c5-49a5-9f27-70b79b0927d0 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:03.145966 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d742f6-7af0-451f-95e8-a89e5d48bad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:03.146646 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d742f6-7af0-451f-95e8-a89e5d48bad4 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:03.146960 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d742f6-7af0-451f-95e8-a89e5d48bad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:03.147254 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d742f6-7af0-451f-95e8-a89e5d48bad4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:03.147972 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d742f6-7af0-451f-95e8-a89e5d48bad4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:03.149902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d742f6-7af0-451f-95e8-a89e5d48bad4 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:03.254962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1421/2849] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:35:03 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:35:03.258937 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31b7e2-5d11-4862-8a1f-93bac401e9f4 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:03.269024 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31b7e2-5d11-4862-8a1f-93bac401e9f4 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:03.330778 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e5e6d-2ae7-4e14-943b-5266060bc5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:03.331627 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e5e6d-2ae7-4e14-943b-5266060bc5ba admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:03.331922 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e5e6d-2ae7-4e14-943b-5266060bc5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:03.332238 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e5e6d-2ae7-4e14-943b-5266060bc5ba admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:03.332905 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e5e6d-2ae7-4e14-943b-5266060bc5ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:03.334853 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e5e6d-2ae7-4e14-943b-5266060bc5ba admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:03.448142 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1429/2850] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:35:03 2022] POST /identity/v3/users => generated 291 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:35:03.451470 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d742f6-7af0-451f-95e8-a89e5d48bad4 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:03.461591 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d742f6-7af0-451f-95e8-a89e5d48bad4 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:03.466194 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ffc111-5767-48da-8f67-255ed8bce6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:03.466864 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ffc111-5767-48da-8f67-255ed8bce6f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:03.467175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ffc111-5767-48da-8f67-255ed8bce6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:03.467451 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ffc111-5767-48da-8f67-255ed8bce6f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:03.468192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ffc111-5767-48da-8f67-255ed8bce6f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:03.469783 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ffc111-5767-48da-8f67-255ed8bce6f2 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:03.480893 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1422/2851] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:35:03.488218 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e5e6d-2ae7-4e14-943b-5266060bc5ba admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:03.496035 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e5e6d-2ae7-4e14-943b-5266060bc5ba admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:03.501028 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4ad24-eee6-4e20-a072-0c0e1120ae1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:03.501702 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4ad24-eee6-4e20-a072-0c0e1120ae1e admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:03.501997 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4ad24-eee6-4e20-a072-0c0e1120ae1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:03.502283 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4ad24-eee6-4e20-a072-0c0e1120ae1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:03.503060 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4ad24-eee6-4e20-a072-0c0e1120ae1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:03.504745 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4ad24-eee6-4e20-a072-0c0e1120ae1e admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:03.514864 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1430/2852] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 09:35:03.519381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ffc111-5767-48da-8f67-255ed8bce6f2 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:03.529344 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ffc111-5767-48da-8f67-255ed8bce6f2 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:03.533847 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea93c36-4415-4d94-ade5-649be645fa0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:03.534719 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea93c36-4415-4d94-ade5-649be645fa0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:03.535026 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea93c36-4415-4d94-ade5-649be645fa0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:03.535305 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea93c36-4415-4d94-ade5-649be645fa0b admin admin] PATH_INFO: `/v3/projects/3a41ed65090b4b21b419e0328349ea3f/users/8d95633d2a2242738e61ee11c90d2edd/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:03.550045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea93c36-4415-4d94-ade5-649be645fa0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a41ed65090b4b21b419e0328349ea3f, user_id=8d95633d2a2242738e61ee11c90d2edd, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:03.552718 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea93c36-4415-4d94-ade5-649be645fa0b admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:03.659459 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1423/2853] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:35:03 2022] PUT /identity/v3/projects/3a41ed65090b4b21b419e0328349ea3f/users/8d95633d2a2242738e61ee11c90d2edd/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:35:03.671572 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63fb9f3-c950-4f33-b8f4-a43457315bbb None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:03.673215 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63fb9f3-c950-4f33-b8f4-a43457315bbb None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:03.673215 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63fb9f3-c950-4f33-b8f4-a43457315bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:03.701802 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-e63fb9f3-c950-4f33-b8f4-a43457315bbb None None] MFA Rules not processed for user `8d95633d2a2242738e61ee11c90d2edd`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:35:03.701802 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-e63fb9f3-c950-4f33-b8f4-a43457315bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:03.777165 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1431/2854] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:35:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:35:03.781556 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea93c36-4415-4d94-ade5-649be645fa0b admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:03.809037 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:35:03.843338 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea93c36-4415-4d94-ade5-649be645fa0b admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:03.848173 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a926379-b029-4ae5-b32e-af9e02866d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:03.848836 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a926379-b029-4ae5-b32e-af9e02866d26 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:03.849125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a926379-b029-4ae5-b32e-af9e02866d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:03.849433 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a926379-b029-4ae5-b32e-af9e02866d26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:03.850190 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a926379-b029-4ae5-b32e-af9e02866d26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:03.852232 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a926379-b029-4ae5-b32e-af9e02866d26 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:03.936923 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1424/2855] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:35:03 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:35:03.940789 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63fb9f3-c950-4f33-b8f4-a43457315bbb None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:03.956669 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63fb9f3-c950-4f33-b8f4-a43457315bbb None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:03.980863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:35:04.007743 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32dcc6-551e-43d5-a973-470acaeb3dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:04.008428 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32dcc6-551e-43d5-a973-470acaeb3dab admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:04.008721 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32dcc6-551e-43d5-a973-470acaeb3dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:04.009008 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32dcc6-551e-43d5-a973-470acaeb3dab admin admin] PATH_INFO: `/v3/users` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:04.009713 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c32dcc6-551e-43d5-a973-470acaeb3dab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:04.012297 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c32dcc6-551e-43d5-a973-470acaeb3dab admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:04.141993 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1432/2856] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:35:03 2022] POST /identity/v3/users => generated 295 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:35:04.146115 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a926379-b029-4ae5-b32e-af9e02866d26 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:04.157131 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a926379-b029-4ae5-b32e-af9e02866d26 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:04.161612 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b5b09f-221b-40fb-9a5e-f684e7472473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:04.162272 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5b09f-221b-40fb-9a5e-f684e7472473 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:04.162559 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5b09f-221b-40fb-9a5e-f684e7472473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:04.162839 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5b09f-221b-40fb-9a5e-f684e7472473 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:04.163602 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b5b09f-221b-40fb-9a5e-f684e7472473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:04.165566 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b5b09f-221b-40fb-9a5e-f684e7472473 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:04.176418 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1425/2857] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:35:04.180603 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32dcc6-551e-43d5-a973-470acaeb3dab admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:04.190667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32dcc6-551e-43d5-a973-470acaeb3dab admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:04.195405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ce0fe-fc2f-4e39-a6c1-5d1d6ae3f596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:04.196279 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ce0fe-fc2f-4e39-a6c1-5d1d6ae3f596 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:04.196567 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ce0fe-fc2f-4e39-a6c1-5d1d6ae3f596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:04.196847 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ce0fe-fc2f-4e39-a6c1-5d1d6ae3f596 admin admin] PATH_INFO: `/v3/projects/25a38dbb1e1e453c8df96fd1457e815b/users/9165a74dde854540a7b78f7b260a13de/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:04.211855 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ce0fe-fc2f-4e39-a6c1-5d1d6ae3f596 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25a38dbb1e1e453c8df96fd1457e815b, user_id=9165a74dde854540a7b78f7b260a13de, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:04.215120 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ce0fe-fc2f-4e39-a6c1-5d1d6ae3f596 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:04.302979 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1433/2858] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:35:04 2022] PUT /identity/v3/projects/25a38dbb1e1e453c8df96fd1457e815b/users/9165a74dde854540a7b78f7b260a13de/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:35:04.306933 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b5b09f-221b-40fb-9a5e-f684e7472473 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:04.317539 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b5b09f-221b-40fb-9a5e-f684e7472473 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:04.373243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edc749-9bc7-42af-b6bc-c523b5c4ea92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:04.373911 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edc749-9bc7-42af-b6bc-c523b5c4ea92 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:04.374200 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edc749-9bc7-42af-b6bc-c523b5c4ea92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:04.374482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edc749-9bc7-42af-b6bc-c523b5c4ea92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:04.375253 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39edc749-9bc7-42af-b6bc-c523b5c4ea92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:04.376878 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39edc749-9bc7-42af-b6bc-c523b5c4ea92 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:04.387373 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1426/2859] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:35:04.391530 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ce0fe-fc2f-4e39-a6c1-5d1d6ae3f596 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:04.401724 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ce0fe-fc2f-4e39-a6c1-5d1d6ae3f596 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:04.406250 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fc2b3-585d-4176-8e74-8516b9e84cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:04.407151 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fc2b3-585d-4176-8e74-8516b9e84cfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:04.407438 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fc2b3-585d-4176-8e74-8516b9e84cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:04.407722 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fc2b3-585d-4176-8e74-8516b9e84cfd admin admin] PATH_INFO: `/v3/projects/25a38dbb1e1e453c8df96fd1457e815b/users/9165a74dde854540a7b78f7b260a13de/roles/9cd080b50c6f4ee8a54bf9ef1655bd83` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:04.409667 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fc2b3-585d-4176-8e74-8516b9e84cfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=25a38dbb1e1e453c8df96fd1457e815b, user_id=9165a74dde854540a7b78f7b260a13de, role_id=9cd080b50c6f4ee8a54bf9ef1655bd83)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:04.412352 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fc2b3-585d-4176-8e74-8516b9e84cfd admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:04.422464 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1434/2860] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:35:04 2022] PUT /identity/v3/projects/25a38dbb1e1e453c8df96fd1457e815b/users/9165a74dde854540a7b78f7b260a13de/roles/9cd080b50c6f4ee8a54bf9ef1655bd83 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:35:04.433311 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8982dce-9c9e-417c-b0d6-db7b1f574387 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:04.433603 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8982dce-9c9e-417c-b0d6-db7b1f574387 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:04.433884 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8982dce-9c9e-417c-b0d6-db7b1f574387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:04.460202 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-a8982dce-9c9e-417c-b0d6-db7b1f574387 None None] MFA Rules not processed for user `9165a74dde854540a7b78f7b260a13de`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:35:04.461395 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-a8982dce-9c9e-417c-b0d6-db7b1f574387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:04.528125 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1427/2861] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:35:04 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:35:04.538809 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fc2b3-585d-4176-8e74-8516b9e84cfd admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:04.540498 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-c22fc2b3-585d-4176-8e74-8516b9e84cfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:04.596253 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fc2b3-585d-4176-8e74-8516b9e84cfd admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:04.599970 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f093ca2-cd4e-4c47-a4b3-102b910bb80f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:04.600611 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f093ca2-cd4e-4c47-a4b3-102b910bb80f service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:04.600894 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f093ca2-cd4e-4c47-a4b3-102b910bb80f service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:04.601169 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f093ca2-cd4e-4c47-a4b3-102b910bb80f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:04.613645 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f093ca2-cd4e-4c47-a4b3-102b910bb80f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:04.615109 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f093ca2-cd4e-4c47-a4b3-102b910bb80f service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:04.625151 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1435/2862] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:35:04 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:35:05.226460 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8982dce-9c9e-417c-b0d6-db7b1f574387 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:05.247504 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:35:05.289552 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8982dce-9c9e-417c-b0d6-db7b1f574387 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:05.294096 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1d7aa-1c77-49ac-aa7c-348f65ef2d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:05.295038 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1d7aa-1c77-49ac-aa7c-348f65ef2d52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:05.295328 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1d7aa-1c77-49ac-aa7c-348f65ef2d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:05.295610 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1d7aa-1c77-49ac-aa7c-348f65ef2d52 admin admin] PATH_INFO: `/v3/users/8d95633d2a2242738e61ee11c90d2edd` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:05.296952 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1d7aa-1c77-49ac-aa7c-348f65ef2d52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d95633d2a2242738e61ee11c90d2edd)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:05.298870 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1d7aa-1c77-49ac-aa7c-348f65ef2d52 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:05.480729 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-2fa1d7aa-1c77-49ac-aa7c-348f65ef2d52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d95633d2a2242738e61ee11c90d2edd'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:05.487541 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-2fa1d7aa-1c77-49ac-aa7c-348f65ef2d52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d95633d2a2242738e61ee11c90d2edd'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:05.488211 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-2fa1d7aa-1c77-49ac-aa7c-348f65ef2d52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d95633d2a2242738e61ee11c90d2edd'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:05.488900 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-2fa1d7aa-1c77-49ac-aa7c-348f65ef2d52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d95633d2a2242738e61ee11c90d2edd'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:05.501474 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-2fa1d7aa-1c77-49ac-aa7c-348f65ef2d52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d95633d2a2242738e61ee11c90d2edd'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:05.583100 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1428/2863] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:35:05 2022] DELETE /identity/v3/users/8d95633d2a2242738e61ee11c90d2edd => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:35:05.587859 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c5fa8-e4e4-45c2-9aeb-a697b971f925 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:05.588376 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c5fa8-e4e4-45c2-9aeb-a697b971f925 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:05.588490 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c5fa8-e4e4-45c2-9aeb-a697b971f925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:05.606461 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-219c5fa8-e4e4-45c2-9aeb-a697b971f925 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:35:05.607718 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-219c5fa8-e4e4-45c2-9aeb-a697b971f925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:05.666425 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1436/2864] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:35:05.676617 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1d7aa-1c77-49ac-aa7c-348f65ef2d52 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:05.678096 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-2fa1d7aa-1c77-49ac-aa7c-348f65ef2d52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:05.735425 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1d7aa-1c77-49ac-aa7c-348f65ef2d52 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:05.739057 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae7f25-a0f0-45db-a3c0-0c78de00c22e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:05.743181 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae7f25-a0f0-45db-a3c0-0c78de00c22e service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:05.743181 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae7f25-a0f0-45db-a3c0-0c78de00c22e service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:05.743181 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae7f25-a0f0-45db-a3c0-0c78de00c22e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:05.750020 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae7f25-a0f0-45db-a3c0-0c78de00c22e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:05.751482 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae7f25-a0f0-45db-a3c0-0c78de00c22e service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:05.760932 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1429/2865] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:35:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:35:06.129056 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c5fa8-e4e4-45c2-9aeb-a697b971f925 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:06.132238 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-219c5fa8-e4e4-45c2-9aeb-a697b971f925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:06.148512 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c5fa8-e4e4-45c2-9aeb-a697b971f925 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:06.154462 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce531ac-1bd4-490a-a2b5-703e5f50c03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:06.155390 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce531ac-1bd4-490a-a2b5-703e5f50c03b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:06.155684 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce531ac-1bd4-490a-a2b5-703e5f50c03b admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:06.155969 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce531ac-1bd4-490a-a2b5-703e5f50c03b admin admin] PATH_INFO: `/v3/projects/3a41ed65090b4b21b419e0328349ea3f` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:06.157340 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce531ac-1bd4-490a-a2b5-703e5f50c03b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a41ed65090b4b21b419e0328349ea3f)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:06.159442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce531ac-1bd4-490a-a2b5-703e5f50c03b admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:06.187424 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:35:06.275228 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4ce531ac-1bd4-490a-a2b5-703e5f50c03b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a41ed65090b4b21b419e0328349ea3f', 'request_id': 'req-4ce531ac-1bd4-490a-a2b5-703e5f50c03b'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:06.281942 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4ce531ac-1bd4-490a-a2b5-703e5f50c03b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a41ed65090b4b21b419e0328349ea3f', 'request_id': 'req-4ce531ac-1bd4-490a-a2b5-703e5f50c03b'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:06.400791 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4ce531ac-1bd4-490a-a2b5-703e5f50c03b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a41ed65090b4b21b419e0328349ea3f', 'request_id': 'req-4ce531ac-1bd4-490a-a2b5-703e5f50c03b'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:06.411962 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4ce531ac-1bd4-490a-a2b5-703e5f50c03b admin admin] The token cache is being invalidate because project 3a41ed65090b4b21b419e0328349ea3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:35:06.412250 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4ce531ac-1bd4-490a-a2b5-703e5f50c03b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:06.412869 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-4ce531ac-1bd4-490a-a2b5-703e5f50c03b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:06.414463 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1437/2866] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:35:06 2022] DELETE /identity/v3/projects/3a41ed65090b4b21b419e0328349ea3f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:35:06.418464 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae7f25-a0f0-45db-a3c0-0c78de00c22e service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:06.428333 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae7f25-a0f0-45db-a3c0-0c78de00c22e service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:06.476789 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:35:06.492995 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e366bf-d1d6-47b5-baa4-857f5cf78b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:06.493868 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e366bf-d1d6-47b5-baa4-857f5cf78b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:06.494154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e366bf-d1d6-47b5-baa4-857f5cf78b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:06.494429 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e366bf-d1d6-47b5-baa4-857f5cf78b28 admin admin] PATH_INFO: `/v3/users/9165a74dde854540a7b78f7b260a13de` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:06.495637 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e366bf-d1d6-47b5-baa4-857f5cf78b28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9165a74dde854540a7b78f7b260a13de)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:06.497607 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e366bf-d1d6-47b5-baa4-857f5cf78b28 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:06.665273 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-53e366bf-d1d6-47b5-baa4-857f5cf78b28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9165a74dde854540a7b78f7b260a13de'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:06.672073 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-53e366bf-d1d6-47b5-baa4-857f5cf78b28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9165a74dde854540a7b78f7b260a13de'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:06.672736 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-53e366bf-d1d6-47b5-baa4-857f5cf78b28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9165a74dde854540a7b78f7b260a13de'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:06.673339 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-53e366bf-d1d6-47b5-baa4-857f5cf78b28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9165a74dde854540a7b78f7b260a13de'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:06.685336 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-53e366bf-d1d6-47b5-baa4-857f5cf78b28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9165a74dde854540a7b78f7b260a13de'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:06.786313 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1430/2867] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:35:06 2022] DELETE /identity/v3/users/9165a74dde854540a7b78f7b260a13de => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:35:07.237022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce531ac-1bd4-490a-a2b5-703e5f50c03b admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:07.238604 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-4ce531ac-1bd4-490a-a2b5-703e5f50c03b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:07.252441 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce531ac-1bd4-490a-a2b5-703e5f50c03b admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:07.309063 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3382db-7e52-40c5-a37b-fa6b45a14140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:07.309930 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3382db-7e52-40c5-a37b-fa6b45a14140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:07.310217 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3382db-7e52-40c5-a37b-fa6b45a14140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:07.310504 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3382db-7e52-40c5-a37b-fa6b45a14140 admin admin] PATH_INFO: `/v3/projects/25a38dbb1e1e453c8df96fd1457e815b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:07.311737 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3382db-7e52-40c5-a37b-fa6b45a14140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25a38dbb1e1e453c8df96fd1457e815b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:07.313698 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3382db-7e52-40c5-a37b-fa6b45a14140 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:07.465369 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-6c3382db-7e52-40c5-a37b-fa6b45a14140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25a38dbb1e1e453c8df96fd1457e815b', 'request_id': 'req-6c3382db-7e52-40c5-a37b-fa6b45a14140'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:07.471653 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-6c3382db-7e52-40c5-a37b-fa6b45a14140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25a38dbb1e1e453c8df96fd1457e815b', 'request_id': 'req-6c3382db-7e52-40c5-a37b-fa6b45a14140'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:07.572090 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-6c3382db-7e52-40c5-a37b-fa6b45a14140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25a38dbb1e1e453c8df96fd1457e815b', 'request_id': 'req-6c3382db-7e52-40c5-a37b-fa6b45a14140'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:07.582744 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-6c3382db-7e52-40c5-a37b-fa6b45a14140 admin admin] The token cache is being invalidate because project 25a38dbb1e1e453c8df96fd1457e815b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:35:07.583065 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-6c3382db-7e52-40c5-a37b-fa6b45a14140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:07.583685 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-6c3382db-7e52-40c5-a37b-fa6b45a14140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:07.585273 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1438/2868] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:35:07 2022] DELETE /identity/v3/projects/25a38dbb1e1e453c8df96fd1457e815b => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 09:35:07.590567 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0880468-c093-4756-895e-ba4a4abc2cd7 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:07.591067 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0880468-c093-4756-895e-ba4a4abc2cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:07.591169 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0880468-c093-4756-895e-ba4a4abc2cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:07.609436 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-a0880468-c093-4756-895e-ba4a4abc2cd7 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:35:07.610626 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-a0880468-c093-4756-895e-ba4a4abc2cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:07.666690 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1431/2869] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:35:07.682148 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4dabed-1b0e-4d77-8f36-ff2f342e0c71 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:07.682493 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4dabed-1b0e-4d77-8f36-ff2f342e0c71 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:07.682721 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4dabed-1b0e-4d77-8f36-ff2f342e0c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:07.701252 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-ce4dabed-1b0e-4d77-8f36-ff2f342e0c71 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:35:07.702454 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ce4dabed-1b0e-4d77-8f36-ff2f342e0c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:07.710222 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1439/2870] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:35:07.714276 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0880468-c093-4756-895e-ba4a4abc2cd7 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:07.725353 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0880468-c093-4756-895e-ba4a4abc2cd7 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:07.730009 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13187770-190d-4df9-a603-174bbad6896a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:07.730673 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13187770-190d-4df9-a603-174bbad6896a admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:07.730996 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13187770-190d-4df9-a603-174bbad6896a admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:07.731280 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13187770-190d-4df9-a603-174bbad6896a admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:07.732010 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13187770-190d-4df9-a603-174bbad6896a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:35:07.732322 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13187770-190d-4df9-a603-174bbad6896a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:07.733930 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13187770-190d-4df9-a603-174bbad6896a admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:07.746038 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1432/2871] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:35:07.750332 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4dabed-1b0e-4d77-8f36-ff2f342e0c71 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:07.759777 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4dabed-1b0e-4d77-8f36-ff2f342e0c71 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:07.764916 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad57910-8e38-4fdc-b59a-99309adf4270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:07.765561 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad57910-8e38-4fdc-b59a-99309adf4270 admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:07.765844 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad57910-8e38-4fdc-b59a-99309adf4270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:07.766179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad57910-8e38-4fdc-b59a-99309adf4270 admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:07.766879 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad57910-8e38-4fdc-b59a-99309adf4270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:07.768815 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad57910-8e38-4fdc-b59a-99309adf4270 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:07.842194 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1440/2872] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:35:07 2022] POST /identity/v3/projects => generated 371 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:35:07.846330 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13187770-190d-4df9-a603-174bbad6896a admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:07.856384 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13187770-190d-4df9-a603-174bbad6896a admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:07.901544 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:35:07.913932 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e741a763-577e-4305-8db6-13dfadbedb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:07.914601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741a763-577e-4305-8db6-13dfadbedb33 admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:07.914890 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741a763-577e-4305-8db6-13dfadbedb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:07.915243 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741a763-577e-4305-8db6-13dfadbedb33 admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:07.915938 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e741a763-577e-4305-8db6-13dfadbedb33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:07.917872 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e741a763-577e-4305-8db6-13dfadbedb33 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:07.971894 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1433/2873] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:35:07 2022] POST /identity/v3/users => generated 286 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:35:07.975871 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad57910-8e38-4fdc-b59a-99309adf4270 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:07.986700 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad57910-8e38-4fdc-b59a-99309adf4270 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:07.991624 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56bf8c-ef41-4b32-9603-680e036b13df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:07.992283 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56bf8c-ef41-4b32-9603-680e036b13df admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:07.992572 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56bf8c-ef41-4b32-9603-680e036b13df admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:07.992860 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56bf8c-ef41-4b32-9603-680e036b13df admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:07.993587 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56bf8c-ef41-4b32-9603-680e036b13df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:07.995218 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56bf8c-ef41-4b32-9603-680e036b13df admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:08.000334 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:35:08.005649 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1441/2874] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:35:08.009475 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e741a763-577e-4305-8db6-13dfadbedb33 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:08.021553 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e741a763-577e-4305-8db6-13dfadbedb33 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:08.026111 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262c4a5-d437-4122-b9b3-ef600cc60cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:08.030073 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262c4a5-d437-4122-b9b3-ef600cc60cde admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:08.030073 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262c4a5-d437-4122-b9b3-ef600cc60cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:08.030073 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262c4a5-d437-4122-b9b3-ef600cc60cde admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:08.030073 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262c4a5-d437-4122-b9b3-ef600cc60cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:08.031055 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262c4a5-d437-4122-b9b3-ef600cc60cde admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:08.041286 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1434/2875] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:35:08.045432 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56bf8c-ef41-4b32-9603-680e036b13df admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:08.055382 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56bf8c-ef41-4b32-9603-680e036b13df admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:08.074876 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5cd20-d95c-4ae3-b0e5-8dbc844815c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:08.074876 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5cd20-d95c-4ae3-b0e5-8dbc844815c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:08.074876 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5cd20-d95c-4ae3-b0e5-8dbc844815c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:08.074876 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5cd20-d95c-4ae3-b0e5-8dbc844815c7 admin admin] PATH_INFO: `/v3/projects/77d1e4d6129e46828a3cb143798d9c80/users/b5751eb1dcbe47fab9367ce3983ff029/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:08.086648 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d5cd20-d95c-4ae3-b0e5-8dbc844815c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d1e4d6129e46828a3cb143798d9c80, user_id=b5751eb1dcbe47fab9367ce3983ff029, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:08.089590 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d5cd20-d95c-4ae3-b0e5-8dbc844815c7 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:08.147671 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1442/2876] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:35:08 2022] PUT /identity/v3/projects/77d1e4d6129e46828a3cb143798d9c80/users/b5751eb1dcbe47fab9367ce3983ff029/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:35:08.212598 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21568ee5-ef39-46da-9281-292869cbaca2 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:08.212919 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21568ee5-ef39-46da-9281-292869cbaca2 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:08.213209 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21568ee5-ef39-46da-9281-292869cbaca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:08.239402 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-21568ee5-ef39-46da-9281-292869cbaca2 None None] MFA Rules not processed for user `b5751eb1dcbe47fab9367ce3983ff029`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:35:08.240620 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-21568ee5-ef39-46da-9281-292869cbaca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:08.306881 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1435/2877] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:35:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:35:08.319143 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5cd20-d95c-4ae3-b0e5-8dbc844815c7 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:08.321034 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-f5d5cd20-d95c-4ae3-b0e5-8dbc844815c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:08.378166 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5cd20-d95c-4ae3-b0e5-8dbc844815c7 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:08.381852 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33488dd0-1699-4419-bff4-2b5113bd6f0c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36a6a3190dad4948a505bbe4c7095699', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:08.382505 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33488dd0-1699-4419-bff4-2b5113bd6f0c service glance] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:08.382793 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33488dd0-1699-4419-bff4-2b5113bd6f0c service glance] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:08.383106 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33488dd0-1699-4419-bff4-2b5113bd6f0c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:08.392084 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33488dd0-1699-4419-bff4-2b5113bd6f0c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:08.393534 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33488dd0-1699-4419-bff4-2b5113bd6f0c service glance] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:08.402558 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1443/2878] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:35:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:35:18.303806 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=44,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:35:18.398342 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=36,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:35:36.860464 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21568ee5-ef39-46da-9281-292869cbaca2 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:36.943951 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21568ee5-ef39-46da-9281-292869cbaca2 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:36.951498 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60931e2d-787b-49fd-a252-6e66aee58d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:36.952638 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60931e2d-787b-49fd-a252-6e66aee58d46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:36.953031 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60931e2d-787b-49fd-a252-6e66aee58d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:36.953413 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60931e2d-787b-49fd-a252-6e66aee58d46 admin admin] PATH_INFO: `/v3/users/b5751eb1dcbe47fab9367ce3983ff029` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:36.954862 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60931e2d-787b-49fd-a252-6e66aee58d46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5751eb1dcbe47fab9367ce3983ff029)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:36.957200 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60931e2d-787b-49fd-a252-6e66aee58d46 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:37.211701 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-60931e2d-787b-49fd-a252-6e66aee58d46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5751eb1dcbe47fab9367ce3983ff029'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:37.219613 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-60931e2d-787b-49fd-a252-6e66aee58d46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5751eb1dcbe47fab9367ce3983ff029'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:37.220331 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-60931e2d-787b-49fd-a252-6e66aee58d46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5751eb1dcbe47fab9367ce3983ff029'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:37.220945 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-60931e2d-787b-49fd-a252-6e66aee58d46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5751eb1dcbe47fab9367ce3983ff029'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:37.234633 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-60931e2d-787b-49fd-a252-6e66aee58d46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5751eb1dcbe47fab9367ce3983ff029'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:37.343600 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1436/2879] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:35:36 2022] DELETE /identity/v3/users/b5751eb1dcbe47fab9367ce3983ff029 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:35:37.348359 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30dea78-c779-4535-ad24-29f1da4eb991 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:37.348850 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30dea78-c779-4535-ad24-29f1da4eb991 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:37.348960 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30dea78-c779-4535-ad24-29f1da4eb991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:37.367523 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-a30dea78-c779-4535-ad24-29f1da4eb991 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:35:37.368758 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a30dea78-c779-4535-ad24-29f1da4eb991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:37.429837 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1444/2880] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:35:37.440025 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60931e2d-787b-49fd-a252-6e66aee58d46 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:37.441428 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-60931e2d-787b-49fd-a252-6e66aee58d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:37.501572 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60931e2d-787b-49fd-a252-6e66aee58d46 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:37.505164 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76726d08-70db-4c0e-af5f-d20a48130571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:37.505849 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76726d08-70db-4c0e-af5f-d20a48130571 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:37.506132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76726d08-70db-4c0e-af5f-d20a48130571 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:37.506408 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76726d08-70db-4c0e-af5f-d20a48130571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:37.516502 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76726d08-70db-4c0e-af5f-d20a48130571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:37.517928 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76726d08-70db-4c0e-af5f-d20a48130571 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:37.527262 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1437/2881] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:35:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 09:35:38.013960 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30dea78-c779-4535-ad24-29f1da4eb991 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:38.015572 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-a30dea78-c779-4535-ad24-29f1da4eb991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:38.028270 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30dea78-c779-4535-ad24-29f1da4eb991 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:38.033051 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553b56d-4e8f-4891-b038-f7e64d430332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:38.034662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553b56d-4e8f-4891-b038-f7e64d430332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:38.034662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553b56d-4e8f-4891-b038-f7e64d430332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:38.034662 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553b56d-4e8f-4891-b038-f7e64d430332 admin admin] PATH_INFO: `/v3/projects/77d1e4d6129e46828a3cb143798d9c80` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:38.035883 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553b56d-4e8f-4891-b038-f7e64d430332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77d1e4d6129e46828a3cb143798d9c80)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:38.037989 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553b56d-4e8f-4891-b038-f7e64d430332 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:38.134320 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-3553b56d-4e8f-4891-b038-f7e64d430332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77d1e4d6129e46828a3cb143798d9c80', 'request_id': 'req-3553b56d-4e8f-4891-b038-f7e64d430332'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:38.140467 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-3553b56d-4e8f-4891-b038-f7e64d430332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77d1e4d6129e46828a3cb143798d9c80', 'request_id': 'req-3553b56d-4e8f-4891-b038-f7e64d430332'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:38.240586 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-3553b56d-4e8f-4891-b038-f7e64d430332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77d1e4d6129e46828a3cb143798d9c80', 'request_id': 'req-3553b56d-4e8f-4891-b038-f7e64d430332'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:38.251796 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-3553b56d-4e8f-4891-b038-f7e64d430332 admin admin] The token cache is being invalidate because project 77d1e4d6129e46828a3cb143798d9c80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:35:38.252088 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-3553b56d-4e8f-4891-b038-f7e64d430332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:38.252711 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-3553b56d-4e8f-4891-b038-f7e64d430332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:38.254284 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1445/2882] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:35:38 2022] DELETE /identity/v3/projects/77d1e4d6129e46828a3cb143798d9c80 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:35:38.259370 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da96c70-1aac-4af1-a5f9-1243fb020a53 None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:38.259872 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da96c70-1aac-4af1-a5f9-1243fb020a53 None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:38.259985 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da96c70-1aac-4af1-a5f9-1243fb020a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:38.278419 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-9da96c70-1aac-4af1-a5f9-1243fb020a53 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:35:38.279659 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-9da96c70-1aac-4af1-a5f9-1243fb020a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:38.296832 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:35:38.340512 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1438/2883] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:35:38.348237 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3a370-5a57-434a-8bb8-7ce6b3894743 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:38.348531 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3a370-5a57-434a-8bb8-7ce6b3894743 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:38.349172 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3a370-5a57-434a-8bb8-7ce6b3894743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:38.367055 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-2af3a370-5a57-434a-8bb8-7ce6b3894743 None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:35:38.368246 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-2af3a370-5a57-434a-8bb8-7ce6b3894743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:38.376839 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1446/2884] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:35:38.381034 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da96c70-1aac-4af1-a5f9-1243fb020a53 None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:38.391214 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da96c70-1aac-4af1-a5f9-1243fb020a53 None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:38.395842 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753c6de-2cb4-4653-baf9-1810c11420b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:38.397508 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753c6de-2cb4-4653-baf9-1810c11420b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:38.397799 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753c6de-2cb4-4653-baf9-1810c11420b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:38.398079 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753c6de-2cb4-4653-baf9-1810c11420b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:38.398808 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753c6de-2cb4-4653-baf9-1810c11420b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80655) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 09:35:38.399153 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753c6de-2cb4-4653-baf9-1810c11420b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:38.400765 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753c6de-2cb4-4653-baf9-1810c11420b6 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:38.412278 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1439/2885] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jun 23 09:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 09:35:38.416283 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3a370-5a57-434a-8bb8-7ce6b3894743 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:38.426340 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3a370-5a57-434a-8bb8-7ce6b3894743 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:38.431044 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f328eb-2d5a-4093-ba4a-30bfdefc52fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:38.431687 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f328eb-2d5a-4093-ba4a-30bfdefc52fd admin admin] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:38.431969 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f328eb-2d5a-4093-ba4a-30bfdefc52fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:38.432244 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f328eb-2d5a-4093-ba4a-30bfdefc52fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:38.432935 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f328eb-2d5a-4093-ba4a-30bfdefc52fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:38.435510 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f328eb-2d5a-4093-ba4a-30bfdefc52fd admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:38.515604 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1447/2886] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jun 23 09:35:38 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:35:38.518722 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753c6de-2cb4-4653-baf9-1810c11420b6 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:38.528778 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753c6de-2cb4-4653-baf9-1810c11420b6 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:38.585161 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8ebc9-97ad-4e75-a800-0ad67f57211e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:38.585843 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8ebc9-97ad-4e75-a800-0ad67f57211e admin admin] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:38.586132 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8ebc9-97ad-4e75-a800-0ad67f57211e admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:38.586412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8ebc9-97ad-4e75-a800-0ad67f57211e admin admin] PATH_INFO: `/v3/users` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:38.587146 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df8ebc9-97ad-4e75-a800-0ad67f57211e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:38.589166 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df8ebc9-97ad-4e75-a800-0ad67f57211e admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:38.674634 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1440/2887] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jun 23 09:35:38 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 09:35:38.678806 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f328eb-2d5a-4093-ba4a-30bfdefc52fd admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:38.689206 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f328eb-2d5a-4093-ba4a-30bfdefc52fd admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:38.693830 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba22ee-f138-4640-8954-de6d19dc4e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:38.694492 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba22ee-f138-4640-8954-de6d19dc4e20 admin admin] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:38.694779 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba22ee-f138-4640-8954-de6d19dc4e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:38.695094 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba22ee-f138-4640-8954-de6d19dc4e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:38.695828 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba22ee-f138-4640-8954-de6d19dc4e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:38.697442 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba22ee-f138-4640-8954-de6d19dc4e20 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:38.708461 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1448/2888] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:35:38.712301 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8ebc9-97ad-4e75-a800-0ad67f57211e admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:38.722841 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8ebc9-97ad-4e75-a800-0ad67f57211e admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:38.727577 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b2060-b949-4268-ad57-13cca0d30ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:38.728235 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b2060-b949-4268-ad57-13cca0d30ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:38.728527 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b2060-b949-4268-ad57-13cca0d30ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:38.728810 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b2060-b949-4268-ad57-13cca0d30ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:38.729536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b2060-b949-4268-ad57-13cca0d30ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:38.731164 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b2060-b949-4268-ad57-13cca0d30ea1 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:38.741236 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1441/2889] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jun 23 09:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 09:35:38.745298 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba22ee-f138-4640-8954-de6d19dc4e20 admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:38.755196 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba22ee-f138-4640-8954-de6d19dc4e20 admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:38.759839 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fe5de-92ff-49a1-a8b2-2d1318852c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:38.760707 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fe5de-92ff-49a1-a8b2-2d1318852c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:38.760989 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fe5de-92ff-49a1-a8b2-2d1318852c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:38.761264 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fe5de-92ff-49a1-a8b2-2d1318852c0f admin admin] PATH_INFO: `/v3/projects/5678ee1cde7045e79ba34b6994ad4637/users/c1d22c04ca814a869911d6c23def1285/roles/53a3d65343394e7caafcc33f12cf138b` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:38.777345 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6fe5de-92ff-49a1-a8b2-2d1318852c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5678ee1cde7045e79ba34b6994ad4637, user_id=c1d22c04ca814a869911d6c23def1285, role_id=53a3d65343394e7caafcc33f12cf138b)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:38.780623 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6fe5de-92ff-49a1-a8b2-2d1318852c0f admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:38.943036 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1449/2890] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jun 23 09:35:38 2022] PUT /identity/v3/projects/5678ee1cde7045e79ba34b6994ad4637/users/c1d22c04ca814a869911d6c23def1285/roles/53a3d65343394e7caafcc33f12cf138b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:35:38.955031 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa210a-1459-4a84-9bda-a3bbd0bb549a None None] REQUEST_METHOD: `POST` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:38.955327 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa210a-1459-4a84-9bda-a3bbd0bb549a None None] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:38.955606 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa210a-1459-4a84-9bda-a3bbd0bb549a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:38.981660 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.auth.core [None req-fbaa210a-1459-4a84-9bda-a3bbd0bb549a None None] MFA Rules not processed for user `c1d22c04ca814a869911d6c23def1285`. Rule list: `[]` (Enabled: `True`). {{(pid=80655) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:35:38.982902 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-fbaa210a-1459-4a84-9bda-a3bbd0bb549a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:39.045592 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1442/2891] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:35:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:35:39.056100 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fe5de-92ff-49a1-a8b2-2d1318852c0f admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:39.057783 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-7a6fe5de-92ff-49a1-a8b2-2d1318852c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:39.080652 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:35:39.114842 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fe5de-92ff-49a1-a8b2-2d1318852c0f admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:39.119313 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f7066-7ce5-48d6-9b8d-8cd665f52440 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:39.119967 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f7066-7ce5-48d6-9b8d-8cd665f52440 service cinder] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:39.120254 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f7066-7ce5-48d6-9b8d-8cd665f52440 service cinder] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:39.120536 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f7066-7ce5-48d6-9b8d-8cd665f52440 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:39.129507 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f7066-7ce5-48d6-9b8d-8cd665f52440 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:39.131027 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f7066-7ce5-48d6-9b8d-8cd665f52440 service cinder] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:39.139908 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1450/2892] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:35:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:35:49.042732 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:35:49.397560 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=17 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:35:50.458861 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa210a-1459-4a84-9bda-a3bbd0bb549a None None] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:50.520788 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa210a-1459-4a84-9bda-a3bbd0bb549a None None] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:50.525366 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94402463-fd3d-4165-a86a-527f9bf41295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:50.526244 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94402463-fd3d-4165-a86a-527f9bf41295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:50.526536 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94402463-fd3d-4165-a86a-527f9bf41295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:50.526820 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94402463-fd3d-4165-a86a-527f9bf41295 admin admin] PATH_INFO: `/v3/users/c1d22c04ca814a869911d6c23def1285` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:50.528045 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94402463-fd3d-4165-a86a-527f9bf41295 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1d22c04ca814a869911d6c23def1285)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:50.530588 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94402463-fd3d-4165-a86a-527f9bf41295 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:50.751657 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-94402463-fd3d-4165-a86a-527f9bf41295 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1d22c04ca814a869911d6c23def1285'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:50.758412 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-94402463-fd3d-4165-a86a-527f9bf41295 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1d22c04ca814a869911d6c23def1285'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:50.759181 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-94402463-fd3d-4165-a86a-527f9bf41295 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1d22c04ca814a869911d6c23def1285'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:50.759797 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-94402463-fd3d-4165-a86a-527f9bf41295 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1d22c04ca814a869911d6c23def1285'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:50.772192 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-94402463-fd3d-4165-a86a-527f9bf41295 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1d22c04ca814a869911d6c23def1285'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:50.857216 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1443/2893] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:35:50 2022] DELETE /identity/v3/users/c1d22c04ca814a869911d6c23def1285 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:35:50.861642 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb330a8-cc3d-48ef-86cb-d2588be648ae None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:50.862127 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb330a8-cc3d-48ef-86cb-d2588be648ae None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:50.862231 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb330a8-cc3d-48ef-86cb-d2588be648ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:50.879622 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-cfb330a8-cc3d-48ef-86cb-d2588be648ae None None] MFA Rules not processed for user `261a7891b0d244a3a8e71c73d6ba80b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:35:50.880815 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-cfb330a8-cc3d-48ef-86cb-d2588be648ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:50.937961 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1451/2894] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:35:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 09:35:50.947531 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94402463-fd3d-4165-a86a-527f9bf41295 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:50.948912 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-94402463-fd3d-4165-a86a-527f9bf41295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:51.005886 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94402463-fd3d-4165-a86a-527f9bf41295 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:51.009549 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9ffbb-62f0-4c4b-b3d3-e3fe9e1d39f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:51.010197 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9ffbb-62f0-4c4b-b3d3-e3fe9e1d39f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:51.010483 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9ffbb-62f0-4c4b-b3d3-e3fe9e1d39f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:51.010760 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9ffbb-62f0-4c4b-b3d3-e3fe9e1d39f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:51.020355 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9ffbb-62f0-4c4b-b3d3-e3fe9e1d39f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:51.021790 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9ffbb-62f0-4c4b-b3d3-e3fe9e1d39f5 service neutron] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:51.031138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1444/2895] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:35:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:35:51.436863 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb330a8-cc3d-48ef-86cb-d2588be648ae None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:35:51.438293 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-cfb330a8-cc3d-48ef-86cb-d2588be648ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:35:51.450842 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb330a8-cc3d-48ef-86cb-d2588be648ae None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:35:51.455928 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d12f6-0ed4-4a84-ace6-5bf5718029a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:35:51.457022 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d12f6-0ed4-4a84-ace6-5bf5718029a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:35:51.457517 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d12f6-0ed4-4a84-ace6-5bf5718029a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:35:51.457801 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d12f6-0ed4-4a84-ace6-5bf5718029a8 admin admin] PATH_INFO: `/v3/projects/5678ee1cde7045e79ba34b6994ad4637` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:35:51.460220 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d12f6-0ed4-4a84-ace6-5bf5718029a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5678ee1cde7045e79ba34b6994ad4637)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:35:51.462123 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d12f6-0ed4-4a84-ace6-5bf5718029a8 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:35:51.565917 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0a7d12f6-0ed4-4a84-ace6-5bf5718029a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5678ee1cde7045e79ba34b6994ad4637', 'request_id': 'req-0a7d12f6-0ed4-4a84-ace6-5bf5718029a8'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:51.572056 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0a7d12f6-0ed4-4a84-ace6-5bf5718029a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5678ee1cde7045e79ba34b6994ad4637', 'request_id': 'req-0a7d12f6-0ed4-4a84-ace6-5bf5718029a8'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:51.648780 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0a7d12f6-0ed4-4a84-ace6-5bf5718029a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5678ee1cde7045e79ba34b6994ad4637', 'request_id': 'req-0a7d12f6-0ed4-4a84-ace6-5bf5718029a8'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:51.659936 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0a7d12f6-0ed4-4a84-ace6-5bf5718029a8 admin admin] The token cache is being invalidate because project 5678ee1cde7045e79ba34b6994ad4637 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:35:51.660229 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0a7d12f6-0ed4-4a84-ace6-5bf5718029a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:51.660858 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-0a7d12f6-0ed4-4a84-ace6-5bf5718029a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:35:51.662407 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1452/2896] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:35:51 2022] DELETE /identity/v3/projects/5678ee1cde7045e79ba34b6994ad4637 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:36:01.026962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:36:01.656890 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:36:44.061634 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9ffbb-62f0-4c4b-b3d3-e3fe9e1d39f5 service neutron] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:36:44.063060 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-44f9ffbb-62f0-4c4b-b3d3-e3fe9e1d39f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:36:44.151468 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9ffbb-62f0-4c4b-b3d3-e3fe9e1d39f5 service neutron] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:36:44.200318 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24569355-c5e9-44df-8d4c-818b08f085fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21a758d0e2645318c89a4fe8805df0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:36:44.201601 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24569355-c5e9-44df-8d4c-818b08f085fc service cinder] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:36:44.201891 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24569355-c5e9-44df-8d4c-818b08f085fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:36:44.202173 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24569355-c5e9-44df-8d4c-818b08f085fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:36:44.204004 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-24569355-c5e9-44df-8d4c-818b08f085fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:36:44.273847 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24569355-c5e9-44df-8d4c-818b08f085fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:36:44.275381 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24569355-c5e9-44df-8d4c-818b08f085fc service cinder] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:36:44.284562 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1445/2897] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:36:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:36:52.726693 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfdf20-0694-4153-a395-147f68090a43 None None] REQUEST_METHOD: `POST` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:36:52.727236 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfdf20-0694-4153-a395-147f68090a43 None None] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:36:52.727354 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfdf20-0694-4153-a395-147f68090a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:36:52.744914 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.auth.core [None req-ebbfdf20-0694-4153-a395-147f68090a43 None None] MFA Rules not processed for user `86a640cdceb34bb9bfbb7b769e797d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 09:36:52.746107 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ebbfdf20-0694-4153-a395-147f68090a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:36:52.752988 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1453/2898] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jun 23 09:36:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 09:36:54.280320 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=87 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:37:02.739366 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=3 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:38:05.354089 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24569355-c5e9-44df-8d4c-818b08f085fc service cinder] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:38:05.355546 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-24569355-c5e9-44df-8d4c-818b08f085fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:38:05.455135 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24569355-c5e9-44df-8d4c-818b08f085fc service cinder] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:38:05.460340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18acc168-f323-4c2d-8669-58a273b2b5b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63b20ceca40e6bceb56eda8ae9686', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:38:05.460993 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acc168-f323-4c2d-8669-58a273b2b5b7 service placement] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:38:05.461281 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acc168-f323-4c2d-8669-58a273b2b5b7 service placement] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:38:05.461560 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acc168-f323-4c2d-8669-58a273b2b5b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:38:05.463442 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-18acc168-f323-4c2d-8669-58a273b2b5b7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:38:05.474857 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18acc168-f323-4c2d-8669-58a273b2b5b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:38:05.476468 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18acc168-f323-4c2d-8669-58a273b2b5b7 service placement] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:38:05.486213 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1446/2899] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:38:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:38:15.482138 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=44 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:38:57.368942 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfdf20-0694-4153-a395-147f68090a43 None None] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:38:57.370737 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-ebbfdf20-0694-4153-a395-147f68090a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:38:57.446633 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfdf20-0694-4153-a395-147f68090a43 None None] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:38:57.450066 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0241981-de5f-4e10-b579-fd9b764a4eb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daed809f511a43ed8d3f88eba10466b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:38:57.450797 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0241981-de5f-4e10-b579-fd9b764a4eb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:38:57.451112 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0241981-de5f-4e10-b579-fd9b764a4eb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:38:57.451389 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0241981-de5f-4e10-b579-fd9b764a4eb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:38:57.453126 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-f0241981-de5f-4e10-b579-fd9b764a4eb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:38:57.534213 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0241981-de5f-4e10-b579-fd9b764a4eb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:38:57.535668 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0241981-de5f-4e10-b579-fd9b764a4eb0 service neutron] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:38:57.555307 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1454/2900] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:38:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:39:01.198723 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18acc168-f323-4c2d-8669-58a273b2b5b7 service placement] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:39:01.200175 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-18acc168-f323-4c2d-8669-58a273b2b5b7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:39:01.213876 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18acc168-f323-4c2d-8669-58a273b2b5b7 service placement] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:39:01.219340 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819f3b0-7d88-41d9-a93a-523e898a10ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63b20ceca40e6bceb56eda8ae9686', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:39:01.220998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819f3b0-7d88-41d9-a93a-523e898a10ac service placement] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:39:01.220998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819f3b0-7d88-41d9-a93a-523e898a10ac service placement] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:39:01.220998 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819f3b0-7d88-41d9-a93a-523e898a10ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:39:01.222616 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-3819f3b0-7d88-41d9-a93a-523e898a10ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:39:01.235371 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819f3b0-7d88-41d9-a93a-523e898a10ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:39:01.236962 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819f3b0-7d88-41d9-a93a-523e898a10ac service placement] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:39:01.247997 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1447/2901] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:39:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:39:04.797999 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0241981-de5f-4e10-b579-fd9b764a4eb0 service neutron] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:39:04.814605 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0241981-de5f-4e10-b579-fd9b764a4eb0 service neutron] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:39:04.824591 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105130f-e99f-4cff-baf6-4a4964f0677c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63b20ceca40e6bceb56eda8ae9686', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:39:04.832470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105130f-e99f-4cff-baf6-4a4964f0677c service placement] REQUEST_METHOD: `GET` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:39:04.832470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105130f-e99f-4cff-baf6-4a4964f0677c service placement] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:39:04.832470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105130f-e99f-4cff-baf6-4a4964f0677c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:39:04.832470 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-1105130f-e99f-4cff-baf6-4a4964f0677c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:39:04.845104 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105130f-e99f-4cff-baf6-4a4964f0677c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:39:04.846746 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105130f-e99f-4cff-baf6-4a4964f0677c service placement] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:39:04.857177 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1455/2902] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:39:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:39:04.933188 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819f3b0-7d88-41d9-a93a-523e898a10ac service placement] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:39:04.935780 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-3819f3b0-7d88-41d9-a93a-523e898a10ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:39:04.950509 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819f3b0-7d88-41d9-a93a-523e898a10ac service placement] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:39:04.963405 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d4886-3344-4d71-abd8-c7046130d948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:39:04.964288 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d4886-3344-4d71-abd8-c7046130d948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:39:04.964573 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d4886-3344-4d71-abd8-c7046130d948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:39:04.964852 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d4886-3344-4d71-abd8-c7046130d948 admin admin] PATH_INFO: `/v3/users/86a640cdceb34bb9bfbb7b769e797d30` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:39:04.966035 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d4886-3344-4d71-abd8-c7046130d948 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86a640cdceb34bb9bfbb7b769e797d30)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:39:04.968062 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d4886-3344-4d71-abd8-c7046130d948 admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:39:05.219473 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ce5d4886-3344-4d71-abd8-c7046130d948 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86a640cdceb34bb9bfbb7b769e797d30'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:39:05.226790 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ce5d4886-3344-4d71-abd8-c7046130d948 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86a640cdceb34bb9bfbb7b769e797d30'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:39:05.227487 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ce5d4886-3344-4d71-abd8-c7046130d948 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86a640cdceb34bb9bfbb7b769e797d30'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:39:05.228101 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ce5d4886-3344-4d71-abd8-c7046130d948 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86a640cdceb34bb9bfbb7b769e797d30'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:39:05.240360 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-ce5d4886-3344-4d71-abd8-c7046130d948 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86a640cdceb34bb9bfbb7b769e797d30'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:39:05.349506 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1448/2903] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:39:04 2022] DELETE /identity/v3/users/86a640cdceb34bb9bfbb7b769e797d30 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:39:05.635644 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105130f-e99f-4cff-baf6-4a4964f0677c service placement] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:39:05.646935 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105130f-e99f-4cff-baf6-4a4964f0677c service placement] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:39:05.691411 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=100 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:39:05.702798 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f427a40-343d-4418-83fc-e14d90b3da2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:39:05.703610 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f427a40-343d-4418-83fc-e14d90b3da2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:39:05.703899 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f427a40-343d-4418-83fc-e14d90b3da2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:39:05.704179 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f427a40-343d-4418-83fc-e14d90b3da2e admin admin] PATH_INFO: `/v3/projects/a100fa3c9d124f9089173a35a1e3691d` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:39:05.705345 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f427a40-343d-4418-83fc-e14d90b3da2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a100fa3c9d124f9089173a35a1e3691d)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:39:05.707296 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f427a40-343d-4418-83fc-e14d90b3da2e admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:39:05.815942 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1f427a40-343d-4418-83fc-e14d90b3da2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a100fa3c9d124f9089173a35a1e3691d', 'request_id': 'req-1f427a40-343d-4418-83fc-e14d90b3da2e'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:39:05.822066 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1f427a40-343d-4418-83fc-e14d90b3da2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a100fa3c9d124f9089173a35a1e3691d', 'request_id': 'req-1f427a40-343d-4418-83fc-e14d90b3da2e'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:39:05.959971 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1f427a40-343d-4418-83fc-e14d90b3da2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a100fa3c9d124f9089173a35a1e3691d', 'request_id': 'req-1f427a40-343d-4418-83fc-e14d90b3da2e'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:39:05.971256 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1f427a40-343d-4418-83fc-e14d90b3da2e admin admin] The token cache is being invalidate because project a100fa3c9d124f9089173a35a1e3691d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:39:05.977743 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1f427a40-343d-4418-83fc-e14d90b3da2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:39:05.977743 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-1f427a40-343d-4418-83fc-e14d90b3da2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:39:05.977743 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1456/2904] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:39:05 2022] DELETE /identity/v3/projects/a100fa3c9d124f9089173a35a1e3691d => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:39:05.978488 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d4886-3344-4d71-abd8-c7046130d948 admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:39:05.979953 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-ce5d4886-3344-4d71-abd8-c7046130d948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:39:05.993025 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d4886-3344-4d71-abd8-c7046130d948 admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:39:06.058134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78b015-ab48-46cf-86ee-663a0cd2af0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:39:06.058134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78b015-ab48-46cf-86ee-663a0cd2af0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:39:06.058134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78b015-ab48-46cf-86ee-663a0cd2af0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:39:06.058134 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78b015-ab48-46cf-86ee-663a0cd2af0a admin admin] PATH_INFO: `/v3/users/ad5ac186e60a4b15bc3ba9efec159ae3` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:39:06.058939 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e78b015-ab48-46cf-86ee-663a0cd2af0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad5ac186e60a4b15bc3ba9efec159ae3)` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:39:06.060328 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e78b015-ab48-46cf-86ee-663a0cd2af0a admin admin] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:39:06.357237 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-5e78b015-ab48-46cf-86ee-663a0cd2af0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad5ac186e60a4b15bc3ba9efec159ae3'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:39:06.364085 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-5e78b015-ab48-46cf-86ee-663a0cd2af0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad5ac186e60a4b15bc3ba9efec159ae3'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:39:06.364764 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-5e78b015-ab48-46cf-86ee-663a0cd2af0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad5ac186e60a4b15bc3ba9efec159ae3'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:39:06.365367 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-5e78b015-ab48-46cf-86ee-663a0cd2af0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad5ac186e60a4b15bc3ba9efec159ae3'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:39:06.369904 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:39:06.377696 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.notifications [None req-5e78b015-ab48-46cf-86ee-663a0cd2af0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad5ac186e60a4b15bc3ba9efec159ae3'} {{(pid=80655) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:39:06.502815 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1449/2905] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jun 23 09:39:05 2022] DELETE /identity/v3/users/ad5ac186e60a4b15bc3ba9efec159ae3 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:39:06.843642 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f427a40-343d-4418-83fc-e14d90b3da2e admin admin] Authenticating user token {{(pid=80654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:39:06.846650 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.fernet_utils [None req-1f427a40-343d-4418-83fc-e14d90b3da2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:39:06.859247 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f427a40-343d-4418-83fc-e14d90b3da2e admin admin] Validating token access rules against request {{(pid=80654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:39:06.916508 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800bee9-9649-4f14-b279-4a6c56fdd272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '261a7891b0d244a3a8e71c73d6ba80b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e5fa9b7cbdd44d0c8b5ffa0c15d81646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:39:06.921116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800bee9-9649-4f14-b279-4a6c56fdd272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:39:06.921116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800bee9-9649-4f14-b279-4a6c56fdd272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:39:06.921116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800bee9-9649-4f14-b279-4a6c56fdd272 admin admin] PATH_INFO: `/v3/projects/a671f8cc8a524a68b9c74e02faa958bd` {{(pid=80654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:39:06.921116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800bee9-9649-4f14-b279-4a6c56fdd272 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a671f8cc8a524a68b9c74e02faa958bd)` {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:39:06.921116 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800bee9-9649-4f14-b279-4a6c56fdd272 admin admin] RBAC: Authorization granted {{(pid=80654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:39:07.037405 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2800bee9-9649-4f14-b279-4a6c56fdd272 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a671f8cc8a524a68b9c74e02faa958bd', 'request_id': 'req-2800bee9-9649-4f14-b279-4a6c56fdd272'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:39:07.044569 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2800bee9-9649-4f14-b279-4a6c56fdd272 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a671f8cc8a524a68b9c74e02faa958bd', 'request_id': 'req-2800bee9-9649-4f14-b279-4a6c56fdd272'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:39:07.137187 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2800bee9-9649-4f14-b279-4a6c56fdd272 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a671f8cc8a524a68b9c74e02faa958bd', 'request_id': 'req-2800bee9-9649-4f14-b279-4a6c56fdd272'} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:39:07.148183 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2800bee9-9649-4f14-b279-4a6c56fdd272 admin admin] The token cache is being invalidate because project a671f8cc8a524a68b9c74e02faa958bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 09:39:07.153451 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2800bee9-9649-4f14-b279-4a6c56fdd272 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:39:07.153451 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG keystone.notifications [None req-2800bee9-9649-4f14-b279-4a6c56fdd272 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 09:39:07.153451 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: [pid: 80654|app: 0|req: 1457/2906] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jun 23 09:39:06 2022] DELETE /identity/v3/projects/a671f8cc8a524a68b9c74e02faa958bd => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 09:39:17.068419 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:DELETE=3,keystone:SELECT=6,keystone:INSERT=1 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:39:17.144949 dv-cinder-847274-1-iscsi devstack@keystone.service[80654]: DEBUG dbcounter [-] [80654] Writing DB stats keystone:SELECT=80,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 09:43:10.316288 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78b015-ab48-46cf-86ee-663a0cd2af0a admin admin] Authenticating user token {{(pid=80655) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 09:43:10.317733 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-5e78b015-ab48-46cf-86ee-663a0cd2af0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:43:10.387053 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78b015-ab48-46cf-86ee-663a0cd2af0a admin admin] Validating token access rules against request {{(pid=80655) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 09:43:10.450079 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cdf75-ca91-4f90-9dbb-0ce67dbdb24d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63b20ceca40e6bceb56eda8ae9686', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93b5040533524222b126bbe93afc56d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80655) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 09:43:10.450737 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cdf75-ca91-4f90-9dbb-0ce67dbdb24d service placement] REQUEST_METHOD: `GET` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 09:43:10.451066 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cdf75-ca91-4f90-9dbb-0ce67dbdb24d service placement] SCRIPT_NAME: `/identity` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 09:43:10.451357 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cdf75-ca91-4f90-9dbb-0ce67dbdb24d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80655) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 09:43:10.453154 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.fernet_utils [None req-b50cdf75-ca91-4f90-9dbb-0ce67dbdb24d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80655) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 09:43:10.464441 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50cdf75-ca91-4f90-9dbb-0ce67dbdb24d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 09:43:10.466054 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50cdf75-ca91-4f90-9dbb-0ce67dbdb24d service placement] RBAC: Authorization granted {{(pid=80655) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 09:43:10.476545 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: [pid: 80655|app: 0|req: 1450/2907] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jun 23 09:43:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 09:43:20.472289 dv-cinder-847274-1-iscsi devstack@keystone.service[80655]: DEBUG dbcounter [-] [80655] Writing DB stats keystone:SELECT=58 {{(pid=80655) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}